-- Logs begin at Thu 2022-09-15 00:04:23 UTC, end at Thu 2022-09-15 01:07:19 UTC. -- Sep 15 00:29:56.795207 dv-cinder-857528-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 15 00:29:56.802515 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 15 00:29:56 2022] *** Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: nodename: dv-cinder-857528-2-iscsi Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: machine: x86_64 Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: clock source: unix Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: pcre jit disabled Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: detected number of CPU cores: 4 Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: current working directory: / Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: detected binary path: /usr/bin/uwsgi-core Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: your processes number limit is 47680 Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: your memory page size is 4096 bytes Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: detected max file descriptor number: 2048 Sep 15 00:29:56.805695 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: lock engine: pthread robust mutexes Sep 15 00:29:56.809882 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: thunder lock: enabled Sep 15 00:29:56.809882 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 15 00:29:56.809882 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 15 00:29:56.845385 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: Python main interpreter initialized at 0x55f9f2a959c0 Sep 15 00:29:56.845385 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: python threads support enabled Sep 15 00:29:56.845385 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: your server socket listen backlog is limited to 100 connections Sep 15 00:29:56.845385 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: your mercy for graceful operations on workers is 90 seconds Sep 15 00:29:56.845714 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: mapped 403077 bytes (393 KB) for 2 cores Sep 15 00:29:56.845821 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: *** Operational MODE: preforking *** Sep 15 00:29:56.846094 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: *** uWSGI is running in multiple interpreter mode *** Sep 15 00:29:56.846094 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: spawned uWSGI master process (pid: 81594) Sep 15 00:29:56.846293 dv-cinder-857528-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 15 00:29:56.848233 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: spawned uWSGI worker 1 (pid: 81608, cores: 1) Sep 15 00:29:56.848233 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: spawned uWSGI worker 2 (pid: 81609, cores: 1) Sep 15 00:29:56.848233 dv-cinder-857528-2-iscsi devstack@keystone.service[81594]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 15 00:30:01.096907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 00:30:01.096907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: from cryptography.utils import int_from_bytes Sep 15 00:30:01.097663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 00:30:01.097663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: from cryptography.utils import int_from_bytes Sep 15 00:30:01.097785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 00:30:01.097785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: from cryptography.utils import int_from_bytes Sep 15 00:30:01.097785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 00:30:01.097785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: from cryptography.utils import int_from_bytes Sep 15 00:30:01.147303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 00:30:01.147446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 00:30:01.147537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 00:30:01.147627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 00:30:01.147689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] command line args: [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 00:30:01.147818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] command line args: [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 00:30:01.147940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 00:30:01.148070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 00:30:01.148156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ================================================================================ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 00:30:01.148259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ================================================================================ {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 00:30:01.148344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] admin_token = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.148514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] admin_token = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.148592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config_dir = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.148751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.148834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] config_dir = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.148934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config_source = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] debug = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] config_source = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] debug = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.149932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_config_append = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_config_append = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.150973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_dir = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_dir = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_file = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.151993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.152933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.153959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.154791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] publish_errors = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] publish_errors = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.155911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] standard_threads = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] standard_threads = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.156954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_journal = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] use_journal = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_json = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] use_json = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_stderr = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] use_stderr = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.157946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_syslog = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.158016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] use_syslog = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.158144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.158222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 00:30:01.158393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.158470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.158629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.158711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.158885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.158963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.159946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.160916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.161863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.162999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.163988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.164994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.165949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.166093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.166171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.166337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.166419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.166582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.167923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.168979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.169983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.170896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.171882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.172982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.173926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.174855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.175869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.176961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.177986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.178924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.179920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.180998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.181993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.182148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.182232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.182931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.183926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.external = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.184987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.external = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.password = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.185972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.password = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.token = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] auth.token = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.186901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187645 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.187873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.188981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.189979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.190928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.191852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.192959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.193993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.194956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.195975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.196867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197480 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.197984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.198934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.199943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.200972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201224 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.201986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.202881 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.203960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204168 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.204964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.205931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.206936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.207970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.208924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.209857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.210971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.211952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.212970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.213949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.214876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.215843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.216838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.217831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.218965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] role.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] role.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.219924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.220963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.221967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.222201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.222283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.222283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.222978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.223913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.224994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.225923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.226949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.227881 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.228866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.229006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.229646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.229772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.229906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.230912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.231875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.232911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.233976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.connection = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.234887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.235934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.236981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.237896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 00:30:01.238950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81609) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 00:30:01.239080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 00:30:01.266185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.266410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.266636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.266844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.267081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.267311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.274887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.275582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.302856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.303082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.303606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.303841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.357849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.358806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.406060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.406286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.409155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.409391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.562855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.563147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.563284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.563522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.563742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.563959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.565877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.566109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.566331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.566549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.566765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.566975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.587243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.587490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.587678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.587762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.587913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.587995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.588135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.588216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.588346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.588426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.589073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.589157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.589283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.589492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.600204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.600855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.613544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.613709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.613940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81609) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.615216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.615438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.615679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.615891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 00:30:01.632209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.632416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 00:30:01.632752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.632880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 00:30:01.633019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.633221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 00:30:01.633308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.633414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 00:30:01.633492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.633686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.633767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.633986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.634072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.634244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.634323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.634551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.644073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.644359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.644633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.644714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.644925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.645008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.645197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.645475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.652568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.653606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.655499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.655796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.656086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.656377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.656668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.656775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.656934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.657062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.657208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.657436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.657515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.657721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.657799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.658007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.658087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.658276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.658353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.658595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.658869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.659197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.659471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.659760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.681588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.683096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.683384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.683680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.685154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.685439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.688113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.688400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.688682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.690123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.690407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.690684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.695046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.695341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.697497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.697786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.701169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.701468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.701755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.702043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.702320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.702596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.702872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.703144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.703822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.704118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.704406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.704694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.704973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.705250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.705518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.705784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.719279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.719581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.722488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.722766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.723459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.726728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.727032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.727149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.731017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.731311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.731829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.734928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.736025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.738201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.738502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.738790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.739075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.739186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.739353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.739673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.742474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.742779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.743063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.743345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.743650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.743933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.755496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.755798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.756068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.756331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.761403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.761689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.761957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.762220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.764005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.765559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.767097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.767372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.769468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.770111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.770404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.770688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.770970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.771065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.771362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.772506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.772783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.775719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.776581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.776877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.777236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.777629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.783135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.783423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.783733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.784015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.784405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.784689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.784976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.789280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.789660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.789939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.790217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.790593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.790882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.791160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.800429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.803349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.806259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.807139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.807769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.809235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.809517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.809920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.810498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.813416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.814955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.816307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.816742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.817085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.817477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.817792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.820710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.821107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.821394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.821671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.824493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.826087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.829012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.829480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.829891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.830032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.830327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.830414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.830609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.837530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81609) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.837817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.838096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.838372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.839246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.839375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.839454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.839607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.839751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.839834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.840017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.840286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.840575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.840846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.841107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.841381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.841656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81609) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.846806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 00:30:01.847091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.847369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.847675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.847946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.848214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.848484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.848758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.849030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.849316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.849585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.849850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.850186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.850596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 00:30:01.875767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f9f2a959c0 pid: 81609 (default app) Sep 15 00:30:01.879028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4fb7d-e298-4ea2-87d0-36ea17eb1114 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:01.879315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4fb7d-e298-4ea2-87d0-36ea17eb1114 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:01.879618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4fb7d-e298-4ea2-87d0-36ea17eb1114 None None] PATH_INFO: `/v3` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:01.884020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Thu Sep 15 00:30:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:01.887107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f9f2a959c0 pid: 81608 (default app) Sep 15 00:30:08.696251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:08.696694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:08.696796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:08.698106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 2/2] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:08.705480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:08.706043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:08.706043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:08.776928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: INFO dbcounter [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] Registered counter for database keystone Sep 15 00:30:08.796081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG oslo_db.sqlalchemy.engines [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81608) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 00:30:08.802193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writer thread running {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 00:30:08.984000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG passlib.handlers.bcrypt [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81608) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 00:30:08.984326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG passlib.handlers.bcrypt [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81608) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 00:30:09.042218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:09.043847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:09.171453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1/3] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 00:30:09.637625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:09.707163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: INFO dbcounter [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] Registered counter for database keystone Sep 15 00:30:09.728408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG oslo_db.sqlalchemy.engines [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81609) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 00:30:09.734587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writer thread running {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 00:30:09.854458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955414e-2f27-45e6-acd3-48e37ef6f158 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:09.858753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:09.859378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:09.859703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:09.859977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:09.868530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:09.920398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 00:30:09.920708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acd119-53fe-48ba-8922-5ca3e5780e87 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:09.930438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 3/4] 10.222.0.15 () {58 vars in 1324 bytes} [Thu Sep 15 00:30:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 00:30:09.936896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:09.952136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5241655-c089-4517-98af-d36347b9d7c0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:09.956565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:09.957705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:09.957981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:09.958248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] PATH_INFO: `/v3/projects/e1704ba0a17f4363ab56f84c09fdea08` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:09.959771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] RBAC: Authorizing `identity:get_project(project_id=e1704ba0a17f4363ab56f84c09fdea08)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:10.013312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 00:30:10.013652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5cad2-ab0e-4b22-bef6-96bb746fa866 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:10.018270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 2/5] 10.222.0.15 () {56 vars in 1207 bytes} [Thu Sep 15 00:30:09 2022] GET /identity/v3/projects/e1704ba0a17f4363ab56f84c09fdea08 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:12.131240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c5c95-3a89-4a38-bd36-c7742b2a5bf1 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:12.131805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c5c95-3a89-4a38-bd36-c7742b2a5bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:12.131805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c5c95-3a89-4a38-bd36-c7742b2a5bf1 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:12.136157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 4/6] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:12.141590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:12.142002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:12.142134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:12.162187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:12.163799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:12.172137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 3/7] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 00:30:12.441281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c5c95-3a89-4a38-bd36-c7742b2a5bf1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:12.454284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c5c95-3a89-4a38-bd36-c7742b2a5bf1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:12.458966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea694d5-ca45-42e0-9da4-fd36acbf0541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:12.459827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea694d5-ca45-42e0-9da4-fd36acbf0541 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:12.460107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea694d5-ca45-42e0-9da4-fd36acbf0541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:12.460377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea694d5-ca45-42e0-9da4-fd36acbf0541 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:12.468832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea694d5-ca45-42e0-9da4-fd36acbf0541 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:12.470595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea694d5-ca45-42e0-9da4-fd36acbf0541 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:12.479400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 5/8] 10.222.0.15 () {58 vars in 1324 bytes} [Thu Sep 15 00:30:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 00:30:12.484850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:12.493748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0446a-9d3b-4183-8114-350bb61d46ac None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:12.498251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c2e3a-34cd-4b4b-9837-e12fdd9fb5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:12.498967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c2e3a-34cd-4b4b-9837-e12fdd9fb5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:12.499325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c2e3a-34cd-4b4b-9837-e12fdd9fb5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:12.499690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c2e3a-34cd-4b4b-9837-e12fdd9fb5b1 admin admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:12.501011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c2e3a-34cd-4b4b-9837-e12fdd9fb5b1 admin admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:12.502834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c2e3a-34cd-4b4b-9837-e12fdd9fb5b1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:12.504425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 4/9] 10.222.0.15 () {56 vars in 1198 bytes} [Thu Sep 15 00:30:12 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:14.714900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4695-e2ac-4ce8-9e6c-b4ad99400ee8 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:14.715411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4695-e2ac-4ce8-9e6c-b4ad99400ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:14.715582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4695-e2ac-4ce8-9e6c-b4ad99400ee8 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:14.716848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 6/10] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:14.722205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:14.722620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:14.722753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:14.740489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:14.741657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:14.804350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5d6c-ad2a-405e-bf30-55bd9ef3027b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:14.804738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5d6c-ad2a-405e-bf30-55bd9ef3027b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:14.805019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5d6c-ad2a-405e-bf30-55bd9ef3027b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:14.806333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 7/11] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:14.811387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:14.811696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:14.811966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:14.821756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 5/12] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:14.855007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG passlib.handlers.bcrypt [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81609) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 00:30:14.855394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG passlib.handlers.bcrypt [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81609) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 00:30:14.905651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:14.907217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:15.007306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 8/13] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:15.090025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.113396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.196208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3de6-08a6-449f-a3b7-c986d4baedd0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.229705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.230386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.230667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.230945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.242280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:15.252003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:15.252949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:15.253916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 6/14] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:15.259268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.272854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.357660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.359971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fcc16-2853-4dcd-8672-661589bbe1b8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.362268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.376393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a976aa2-6f99-4909-87bd-5f17ec1bbbcc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.398324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.399184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.399469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.399776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.400686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:15.401025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.402463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.410510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.411112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.411390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.411689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.414555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 9/15] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:15.420737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.484332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.485810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.517576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efdce8-0a2e-41b6-9621-a1e9188be133 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.550938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.551826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.552095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.552532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.554013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.555615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.557733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 10/16] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:15.563215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.568223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.586593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 7/17] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 00:30:15.592647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.598043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.664324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c434025-1b00-4fbf-b57a-dffa0cd7e761 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.694935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb273a1-d9c2-47de-a7a1-b584d20d9ceb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.697948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.698583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.698859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.699127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.700260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.701695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.703253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 11/18] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:15.708888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.729827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.730560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.730922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.731200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.741430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.742903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:30:15.763361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:15.764353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:30:15.765161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:30:15.765161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:30:15.765161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:15.765161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 8/19] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:30:15.770253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:15.804067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.808442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f05b86-2450-418c-aa4d-05f154c2b87b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.809941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:15.843634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.844245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.844518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.844803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.845507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.domain.id=default, user.id=34b8b29876a54a4285fc37bd5edb4042) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:15.845812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.847243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11d68-dd43-4548-a189-ce3b58d2fc45 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.856221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 12/20] 10.222.0.15 () {56 vars in 1322 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/role_assignments?user.id=34b8b29876a54a4285fc37bd5edb4042&scope.domain.id=default&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:15.870597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6584ec-fcf1-4c44-8af3-7215e0a90c03 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:15.906923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:15.907584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:15.907863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:15.908132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:15.908803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:15.909105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:15.910549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4121e26-d549-426f-b47d-cdaffd477a0a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:15.929761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 9/21] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:18.640473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bda0f-2a97-47bd-801d-6142b5397aad None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:18.640473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bda0f-2a97-47bd-801d-6142b5397aad None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:18.640473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bda0f-2a97-47bd-801d-6142b5397aad None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:18.641386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 13/22] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:18.644699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ce08-3907-4d02-8a5c-9d1f768ef9bd None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:18.645092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ce08-3907-4d02-8a5c-9d1f768ef9bd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:18.645266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ce08-3907-4d02-8a5c-9d1f768ef9bd None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:18.646987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 10/23] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:18.648507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cc1c5-3685-45d2-b953-5d3fa6b7d439 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:18.649002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cc1c5-3685-45d2-b953-5d3fa6b7d439 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:18.649582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cc1c5-3685-45d2-b953-5d3fa6b7d439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:18.659212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7b2e-9429-40f1-be40-802b470cc610 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:18.659505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7b2e-9429-40f1-be40-802b470cc610 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:18.659800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7b2e-9429-40f1-be40-802b470cc610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:18.692503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d94cc1c5-3685-45d2-b953-5d3fa6b7d439 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:18.694040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d94cc1c5-3685-45d2-b953-5d3fa6b7d439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:18.703515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4c2e7b2e-9429-40f1-be40-802b470cc610 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:18.703515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4c2e7b2e-9429-40f1-be40-802b470cc610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:18.832290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 14/24] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:18.834765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba70a31-2125-41dd-84c0-f11c980d1646 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:18.835057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba70a31-2125-41dd-84c0-f11c980d1646 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:18.835332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba70a31-2125-41dd-84c0-f11c980d1646 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:18.840332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 15/25] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:18.842002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:18.842288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:18.842563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:18.868596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 11/26] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:18.875192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:18.876695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:18.961962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 16/27] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:19.276717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37fce8-381a-4444-91d9-3f0cd49d05ba None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:19.277381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37fce8-381a-4444-91d9-3f0cd49d05ba None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:19.277862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37fce8-381a-4444-91d9-3f0cd49d05ba None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:19.279411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 12/28] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:19.283894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:19.285054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:19.285546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:19.286049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:19.318688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:19.319918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:19.360818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:19.380984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:19.467281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 13/29] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:19.468448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:19.469397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378c3e-dbb4-4150-be62-0c10f76e1ba9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:19.508431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:19.509297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:19.509745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:19.510757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:19.546011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:19.547854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:30:19.581039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:19.581988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:30:19.582781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:30:19.582781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:30:19.582781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:19.583052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 17/30] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:19.590328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:19.655597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b11d10-434c-4a7f-97ac-c381ed923e00 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:19.725697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:19.727736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:19.728244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:19.729985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:19.730818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:19.732461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:19.758619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:19.814289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057011e-3cb0-4da8-ad61-b9eed12b05cb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:19.849610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:19.850494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:19.850912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:19.851320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:19.863174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:19.870391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:19.871348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:19.872330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:19.874515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 18/31] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:19.875595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:19.923509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 14/32] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 00:30:19 2022] POST /identity/v3/roles => generated 219 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:19.923509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:19.953187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:20.056660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dad7e9-b903-4da0-8521-838c13a13edb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.097666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c222a-95a4-415f-a93b-492c870f75e1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.100271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.101157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.101667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.102134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.103104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:20.103716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.105372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.126030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:20.134934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 19/33] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:20.136726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.159862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37d541-8ec2-460d-8d40-859cfcc2b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.160829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d541-8ec2-460d-8d40-859cfcc2b42b None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.161111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d541-8ec2-460d-8d40-859cfcc2b42b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.161387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d541-8ec2-460d-8d40-859cfcc2b42b None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.162526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37d541-8ec2-460d-8d40-859cfcc2b42b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.164122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37d541-8ec2-460d-8d40-859cfcc2b42b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.182838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 15/34] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 00:30:19 2022] POST /identity/v3/roles => generated 225 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:20.185198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229a5e-cf5c-451c-bb53-8ec6b68a3cd4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.186095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229a5e-cf5c-451c-bb53-8ec6b68a3cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.186524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229a5e-cf5c-451c-bb53-8ec6b68a3cd4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.189327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 16/35] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:20.190676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229a5e-cf5c-451c-bb53-8ec6b68a3cd4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.276488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ea1e4-0231-488e-a764-55daa9ca6837 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.297678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:20.318725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.319348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.319672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.319950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.320656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:20.320936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.322349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f647fa2-afdc-4fa0-9216-e414790e08a1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.332253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 20/36] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:20.332931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229a5e-cf5c-451c-bb53-8ec6b68a3cd4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.338661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894d91d-eb80-4a1f-807c-8d7f572c291e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.340194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894d91d-eb80-4a1f-807c-8d7f572c291e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.340194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894d91d-eb80-4a1f-807c-8d7f572c291e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.341011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 21/37] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:20.341925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc2ad-bb7c-431c-a325-39bce65336a8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.342220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc2ad-bb7c-431c-a325-39bce65336a8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.342486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc2ad-bb7c-431c-a325-39bce65336a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.363951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cc4fc2ad-bb7c-431c-a325-39bce65336a8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:20.365105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cc4fc2ad-bb7c-431c-a325-39bce65336a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:20.379916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.381053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.381053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.381053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.381719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.384509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.418203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 17/38] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:30:20 2022] POST /identity/v3/projects => generated 301 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:20.419413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.444033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:20.461457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 22/39] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:20.463954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.464420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.464972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.485417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:20.487160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:20.548117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d30f-5465-4207-bfa3-bcaba5242a55 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.566801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 23/40] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:20.593308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:20.603840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.604960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.605297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.605588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.606757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.608437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.610273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 18/41] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:20 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:20.616205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.737479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e945377-ae13-4902-9f02-d7f485bf8c06 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.743762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.761602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:20.778818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.779462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.779935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.780219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.782097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.783574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.789717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 24/42] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:20.791156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.849652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fb3a-8c34-4b75-88a8-106cd3da9419 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.883428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.884006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.884286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.884560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.885274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.886691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.895850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7327-3dc0-4ed1-a89d-7698da2f3deb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:20.905130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 19/43] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 00:30:20 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:30:20.906131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:20.938003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:20.938686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:20.938965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:20.939327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] PATH_INFO: `/v3/domains/default/users/34b8b29876a54a4285fc37bd5edb4042/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:20.941325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=34b8b29876a54a4285fc37bd5edb4042, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:20.942807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:20.970749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:21.029897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b2db-4dcb-489e-89c7-fa93de215cad None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:21.063602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 25/44] 10.222.0.15 () {58 vars in 1361 bytes} [Thu Sep 15 00:30:20 2022] PUT /identity/v3/domains/default/users/34b8b29876a54a4285fc37bd5edb4042/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:21.071170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7525b-1dfa-4c23-bd1a-f2c95ec007e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:21.072939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7525b-1dfa-4c23-bd1a-f2c95ec007e7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:21.073448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7525b-1dfa-4c23-bd1a-f2c95ec007e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:21.073916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7525b-1dfa-4c23-bd1a-f2c95ec007e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:21.075452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7525b-1dfa-4c23-bd1a-f2c95ec007e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:21.077001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7525b-1dfa-4c23-bd1a-f2c95ec007e7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:21.078976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 20/45] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:21.085255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:21.151592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:21.205181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bedfbe-59e4-4f75-8176-61035f47aa2c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:21.246332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b80779-181c-4de3-8015-86978bde73b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:21.246957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80779-181c-4de3-8015-86978bde73b4 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:21.247245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80779-181c-4de3-8015-86978bde73b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:21.247568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80779-181c-4de3-8015-86978bde73b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:21.248240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b80779-181c-4de3-8015-86978bde73b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:21.249890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b80779-181c-4de3-8015-86978bde73b4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:21.282741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 26/46] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:30:21 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:23.386393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac234c7-108d-4c17-ab98-b917295b27ef None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:23.386925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac234c7-108d-4c17-ab98-b917295b27ef None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:23.387038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac234c7-108d-4c17-ab98-b917295b27ef None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:23.388347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 21/47] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:23.394217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97122f-3819-4d94-a3aa-37a60f4463f2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:23.394642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97122f-3819-4d94-a3aa-37a60f4463f2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:23.394767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97122f-3819-4d94-a3aa-37a60f4463f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:23.413051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7f97122f-3819-4d94-a3aa-37a60f4463f2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:23.414652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7f97122f-3819-4d94-a3aa-37a60f4463f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:23.481191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8a84b-4d97-4c6c-b06b-393b0bf7e63a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:23.481501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8a84b-4d97-4c6c-b06b-393b0bf7e63a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:23.481896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8a84b-4d97-4c6c-b06b-393b0bf7e63a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:23.482901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 27/48] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:23.483230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 22/49] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:23.488619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:23.488904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:23.489180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:23.507379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:23.508854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:23.532240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:23.576754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 28/50] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:23.751750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8a84b-4d97-4c6c-b06b-393b0bf7e63a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:23.842259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:23.967298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8a84b-4d97-4c6c-b06b-393b0bf7e63a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:23.992999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:24.004804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.005436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.005718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.005990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.016350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:24.021262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:24.022211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:24.022960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 23/51] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:24.029093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:24.047834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb863a0-a1a6-4b18-bc6c-5162c57fd0c2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:24.079993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.080620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.080894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.081165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.082258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:24.083729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:24.085423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 29/52] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:24.092639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:24.120188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:24.128930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ebfec-a102-43da-90e8-e04969337134 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:24.163690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.164289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.164566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.164836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.165499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:24.165798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:24.167767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:24.180165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 24/53] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:24.185907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:24.189915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae7d97-dfb3-4eb5-ba5b-06eaf96ca96e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:24.224012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.224551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.224829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.225100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.225742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:24.227165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:24.231398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:24.254297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 30/54] 10.222.0.15 () {60 vars in 1140 bytes} [Thu Sep 15 00:30:24 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:30:24.291622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6b751-1448-4244-a978-43ce57377e26 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:24.327505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.328163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.328439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.328709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.329900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:24.331405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:24.333069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 25/55] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:24 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:24.339308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:24.424091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:24.444614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8bd2a-6296-49f9-b114-4e359626a319 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:24.480430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947203a7-bf96-47b9-a4d5-aa53f263f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.481007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947203a7-bf96-47b9-a4d5-aa53f263f39d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.481290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947203a7-bf96-47b9-a4d5-aa53f263f39d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.481567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947203a7-bf96-47b9-a4d5-aa53f263f39d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.482673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947203a7-bf96-47b9-a4d5-aa53f263f39d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:24.484171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947203a7-bf96-47b9-a4d5-aa53f263f39d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:24.485750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 31/56] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:24.491653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:24.590266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba27c2f-1c4f-41f9-b50b-e8e33470049a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:24.602938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:24.624138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:24.624829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:24.625180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:24.625515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:24.626306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.domain.id=default, user.id=34b8b29876a54a4285fc37bd5edb4042) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:24.626681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:24.628214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da626c07-178a-4245-8c31-c3b5b9042a29 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:24.636270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 26/57] 10.222.0.15 () {56 vars in 1322 bytes} [Thu Sep 15 00:30:24 2022] GET /identity/v3/role_assignments?user.id=34b8b29876a54a4285fc37bd5edb4042&scope.domain.id=default&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:26.410105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fa767-cd28-4d4e-b457-2a84666c5dc7 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:26.410613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fa767-cd28-4d4e-b457-2a84666c5dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:26.410693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fa767-cd28-4d4e-b457-2a84666c5dc7 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:26.412025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 32/58] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:26.417692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:26.418056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:26.418250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:26.437206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:26.438732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:26.502839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 27/59] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:26.782043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fa767-cd28-4d4e-b457-2a84666c5dc7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:26.879259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fa767-cd28-4d4e-b457-2a84666c5dc7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:26.912174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92fd2-6b7f-4cff-b72f-d3a1dbfe58fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:26.912796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92fd2-6b7f-4cff-b72f-d3a1dbfe58fb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:26.913080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92fd2-6b7f-4cff-b72f-d3a1dbfe58fb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:26.913359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92fd2-6b7f-4cff-b72f-d3a1dbfe58fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:26.914469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92fd2-6b7f-4cff-b72f-d3a1dbfe58fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:26.915954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92fd2-6b7f-4cff-b72f-d3a1dbfe58fb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:26.917577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 33/60] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:26.923077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:27.023000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eef68a-22a9-44ff-80da-fd3b5e2e5d56 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:27.030038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:27.056057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02045dd5-a358-4c31-9e8f-d2652fc47c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:27.056669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02045dd5-a358-4c31-9e8f-d2652fc47c24 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:27.057479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02045dd5-a358-4c31-9e8f-d2652fc47c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:27.057756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02045dd5-a358-4c31-9e8f-d2652fc47c24 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:27.058402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02045dd5-a358-4c31-9e8f-d2652fc47c24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:27.059873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02045dd5-a358-4c31-9e8f-d2652fc47c24 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:27.086051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 28/61] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:30:26 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:30.352226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1f064-3565-43c8-884d-5e4625c9732f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:30.352756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1f064-3565-43c8-884d-5e4625c9732f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:30.352756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1f064-3565-43c8-884d-5e4625c9732f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:30.354923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 34/62] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:30.360652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53572c34-3df2-418d-8de6-40933280f58b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:30.361077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53572c34-3df2-418d-8de6-40933280f58b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:30.361201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53572c34-3df2-418d-8de6-40933280f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:30.381240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-53572c34-3df2-418d-8de6-40933280f58b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:30.382448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-53572c34-3df2-418d-8de6-40933280f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:30.505734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 29/63] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:30.869162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d3bc-d18c-4f37-838f-7d5f8686856f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:30.869977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d3bc-d18c-4f37-838f-7d5f8686856f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:30.870236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d3bc-d18c-4f37-838f-7d5f8686856f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:30.871611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 35/64] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:30.880193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61847a1-7903-4a62-9cba-0902bbcf03f3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:30.880193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61847a1-7903-4a62-9cba-0902bbcf03f3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:30.880193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61847a1-7903-4a62-9cba-0902bbcf03f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:30.894424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3d3bc-d18c-4f37-838f-7d5f8686856f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:30.911740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d61847a1-7903-4a62-9cba-0902bbcf03f3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:30.914169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d61847a1-7903-4a62-9cba-0902bbcf03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:30.929866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:31.048383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3d3bc-d18c-4f37-838f-7d5f8686856f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:31.049121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 30/65] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:31.050541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f713054-bc73-4d33-a296-bec2dd968914 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.050991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f713054-bc73-4d33-a296-bec2dd968914 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.051400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f713054-bc73-4d33-a296-bec2dd968914 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.052843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 31/66] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:31.065002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.065536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.065942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.089739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:31.090713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.091179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.091700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.103196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:31.109419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:31.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:31.111009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:31.112092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:31.113422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 36/67] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:30:31.119121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:31.129508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:31.160169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:31.250335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 32/68] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:31.281248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:31.309089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02016c90-b9b5-4559-8af2-d5f0e3b8133a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:31.357094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:31.358512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.359014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.359572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.360472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:31.360973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:31.362610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa17c1-601c-4cbe-98f6-5fe66cbcd27a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:31.380366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 37/69] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:31.384594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:31.480739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cdc09-aef0-4c64-b2f4-d6ae14d9112e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.481640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cdc09-aef0-4c64-b2f4-d6ae14d9112e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.482106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cdc09-aef0-4c64-b2f4-d6ae14d9112e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.492019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 38/70] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:31.499569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.500095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.500605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.535853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:31.541013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:31.592964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5cc97-3fbd-4821-b999-bc8e988c6d21 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:31.635151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cfc17-d0da-406e-b2e6-6a9c989773aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:31.635869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cfc17-d0da-406e-b2e6-6a9c989773aa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.636152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cfc17-d0da-406e-b2e6-6a9c989773aa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.636430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cfc17-d0da-406e-b2e6-6a9c989773aa None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.643019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 39/71] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:31.643750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:31.656497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cfc17-d0da-406e-b2e6-6a9c989773aa None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:31.660143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cfc17-d0da-406e-b2e6-6a9c989773aa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:31.661850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 33/72] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:31 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:31.663056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1426bac-357e-45fa-ac13-3eff54da4914 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.663335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1426bac-357e-45fa-ac13-3eff54da4914 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.664237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1426bac-357e-45fa-ac13-3eff54da4914 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.665513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 34/73] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:31.668378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1426bac-357e-45fa-ac13-3eff54da4914 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:31.683859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:31.739431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:31.786565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568488ac-3383-4c57-bdcd-a97283aee5d5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:31.792649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1426bac-357e-45fa-ac13-3eff54da4914 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:31.829721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9508fb1-3fe4-4320-ae3d-f51a46036aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:31.830808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9508fb1-3fe4-4320-ae3d-f51a46036aeb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.831324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9508fb1-3fe4-4320-ae3d-f51a46036aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.831791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9508fb1-3fe4-4320-ae3d-f51a46036aeb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.832256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:31.832923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.833209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.833487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.845781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:31.846202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9508fb1-3fe4-4320-ae3d-f51a46036aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:31.850666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:31.851537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:31.852304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-c9508fb1-3fe4-4320-ae3d-f51a46036aeb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:31.852897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:31.853691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:31.854449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:31.856469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 35/74] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:31.856469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:31.859369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 40/75] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:30:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:31.860924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:31.861354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:31.861634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:31.882895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:31.884747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:31.944157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:31.961578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 41/76] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:31.962678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:31.987355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7eebf-0a4a-460b-80f1-c6e9d8380123 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.024004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.038317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.039152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.039621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.040051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.082551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0c7f-7e1e-4e4b-9c2d-5d0d25bc8d49 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.132322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.134056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.136876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.138329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.138609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.138882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.139585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:32.139894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.141276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.155989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 42/77] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.155989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.234930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.269070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072700-f80c-418f-94a6-37a931846af5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.271989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 36/78] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:32.272633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.296483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.312264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.312885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.313165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.313439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.314637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:32.314945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.316393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.325262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 43/79] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.326074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.382451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82773-36b6-4fc1-8d4d-727ba2fb4825 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.420332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.421753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.422203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.422617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.433582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b99dd-f72b-4f6e-897f-df5b07aa5f1f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.434354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:32.439343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:32.440412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:32.441215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:32.443157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 37/80] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:32.444115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.474734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.475428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.475754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.476029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.477174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.479158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.485224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 44/81] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.485224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.486353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.543840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.564893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd26041-5b88-4e88-84dd-2941fdf878ac None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.598591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81ba51-6a7c-4c5f-9e68-065981c25de1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.604699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.605405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.605754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.606095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.607354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.608871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.610618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 38/82] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.611504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.643373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.644102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.644387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.644777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.646688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.648335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.650001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 45/83] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.655072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.726472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.732054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df0f9-5f72-4308-abe2-fcb06f2d3663 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.763233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ca947-58cc-4ee7-b7a2-1bfa99648a8e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.768495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.769131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.769418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.769697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.771079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.772597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.774490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 39/84] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.775286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.786578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.806093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.807336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.807714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.808050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.808802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:32.809170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.810814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.819439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 46/85] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:32.820040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:32.883538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3736068-c497-4df0-91a1-7a2a404e3963 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.922289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440357a1-cc26-40d6-a1f4-1f1e08fa77e0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:32.923153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.923883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.926064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.926400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.960103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:32.960814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:32.961268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:32.961563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:32.962287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:32.964426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:32.965095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:32.995160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 47/86] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:30:32 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:32.996241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:33.010274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:33.012131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:33.049989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:33.146451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712496-e4a0-490b-84cd-7672b22719b4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:33.163461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 40/87] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:33.165058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:33.190463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:33.191132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:33.191423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:33.191732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:33.192481 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=ce85ea6bdfa54b3298ecf834a6d7fd47) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:33.192789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:33.194245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:33.204380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 48/88] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:32 2022] GET /identity/v3/role_assignments?user.id=ce85ea6bdfa54b3298ecf834a6d7fd47&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=87f1831257824560813f954348534540 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:33.205652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:33.271575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:33.346768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559be416-0487-4a9b-96cc-b213cc2588fc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:33.412610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:33.414556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:33.414556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:33.414556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:33.414556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:33.424900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6a5d8-67e7-44c6-a816-0a35f3380ad1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:33.504586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:33.505279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:33.505533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:33.505751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:33.506124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:33.507281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:33.508361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:33.509906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:33.513482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 49/89] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:33 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:33.514348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:33.686895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:33.698772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f87070-7cdd-4444-b68c-1d2a522078ff None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:33.760936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 41/90] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:33.761847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:33.788344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:33.788998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:33.789281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:33.789556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:33.799552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:33.799552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:33.799552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 50/91] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:33 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:33.799552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:33.851877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:33.986962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae36fc-181e-4b95-88a7-cbd258ae9db4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:33.990476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76cdac-af93-4f9e-88ae-37998bb15647 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:34.062938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:34.063613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:34.063902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:34.064181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:34.066245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:34.066867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:34.067177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:34.067453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:34.068691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:34.071341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:34.072078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 51/92] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:33 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:34.073357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:34.146762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:34.245746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374997f6-1fa0-4301-bfa4-61bdcaa9a367 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:34.248716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:34.249416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:34.250918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:34.333482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:34.334203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:34.334483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:34.334751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:34.335508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=34b8b29876a54a4285fc37bd5edb4042) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:34.335821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:34.337272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:34.361509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 52/93] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:34 2022] GET /identity/v3/role_assignments?user.id=34b8b29876a54a4285fc37bd5edb4042&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:34.362394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:34.450825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 42/94] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:34.460598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:34.608644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:34.675113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd091-8403-47fa-9cd0-e61c844a1fd3 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:34.693226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea381b-90ee-4259-9919-945fedcc0c65 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:34.740741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:34.741979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:34.742444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:34.742918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:34.744758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] RBAC: Adding query filter params (role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=ce85ea6bdfa54b3298ecf834a6d7fd47) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:34.745329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:34.747087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:34.759409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 53/95] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:34 2022] GET /identity/v3/role_assignments?user.id=ce85ea6bdfa54b3298ecf834a6d7fd47&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=1d2b7e763c90438eab518a55d7589260 => generated 259 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:34.777933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3969c3-9386-47e9-b6fe-b620f1c9822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:34.778597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3969c3-9386-47e9-b6fe-b620f1c9822d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:34.778879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3969c3-9386-47e9-b6fe-b620f1c9822d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:34.779155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3969c3-9386-47e9-b6fe-b620f1c9822d None admin] PATH_INFO: `/v3/projects/1b1bd105fca54dde86786cbabad26012` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:34.784928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3969c3-9386-47e9-b6fe-b620f1c9822d None admin] RBAC: Authorizing `identity:get_project(project_id=1b1bd105fca54dde86786cbabad26012)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:34.786461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3969c3-9386-47e9-b6fe-b620f1c9822d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:34.788295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 43/96] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:34 2022] GET /identity/v3/projects/1b1bd105fca54dde86786cbabad26012 => generated 312 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:34.796182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:34.859947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:34.963622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bdfa3-fc34-42f9-b461-83bb45238490 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:35.024405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:35.025424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:35.025910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:35.026378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:35.027334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, scope.project.id=1b1bd105fca54dde86786cbabad26012, user.id=ce85ea6bdfa54b3298ecf834a6d7fd47) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:35.027924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:35.029614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6284e53-e594-4939-8050-2d466ef13246 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:35.043212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 54/97] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:34 2022] GET /identity/v3/role_assignments?user.id=ce85ea6bdfa54b3298ecf834a6d7fd47&scope.project.id=1b1bd105fca54dde86786cbabad26012&role.id=87f1831257824560813f954348534540 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:36.089621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f0870-bc00-494d-a5c1-5858c1af6713 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.090143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f0870-bc00-494d-a5c1-5858c1af6713 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.090143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f0870-bc00-494d-a5c1-5858c1af6713 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.091619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 44/98] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:36.098762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9284ae9-4e42-4fdd-9427-ecd30e2b6fdb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.098762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9284ae9-4e42-4fdd-9427-ecd30e2b6fdb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.098762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9284ae9-4e42-4fdd-9427-ecd30e2b6fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.132741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a9284ae9-4e42-4fdd-9427-ecd30e2b6fdb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:36.134572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a9284ae9-4e42-4fdd-9427-ecd30e2b6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:36.234503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 55/99] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:36.284648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751aba-4afd-4fb0-889d-23ec45637562 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.284964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751aba-4afd-4fb0-889d-23ec45637562 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.285198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751aba-4afd-4fb0-889d-23ec45637562 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.286547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 45/100] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:36.293271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcb6ed-0fee-4708-beb2-6284d628b420 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.293271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcb6ed-0fee-4708-beb2-6284d628b420 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.293271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcb6ed-0fee-4708-beb2-6284d628b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.321776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-f7dcb6ed-0fee-4708-beb2-6284d628b420 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:36.323336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f7dcb6ed-0fee-4708-beb2-6284d628b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:36.363599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:36.427213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 56/101] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:36.667308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751aba-4afd-4fb0-889d-23ec45637562 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:36.707628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d749e-4db3-4180-bf56-3e1096e78f86 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.707915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d749e-4db3-4180-bf56-3e1096e78f86 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.708348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d749e-4db3-4180-bf56-3e1096e78f86 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.709850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 57/102] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:36.717178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.717774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.718181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.753895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:36.755581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:36.909526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751aba-4afd-4fb0-889d-23ec45637562 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:36.959077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:36.962014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:36.962465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:36.963382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:36.968644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:36.970259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:36.972118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 46/103] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:36.980401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:36.985969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 58/104] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:36.986388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:36.987154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:37.172368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:37.197752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf91083-3e96-4e85-af92-f0fa6ea543b5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:37.213246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7bed6-6f66-4538-923d-98f8f80dbac0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:37.260414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:37.261070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:37.261358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:37.261634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:37.287252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:37.301580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:37.302468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:37.303251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 47/105] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:37.311739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:37.344456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:37.345357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:37.345744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:37.346122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:37.346883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:37.348566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:37.392617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:37.427510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 59/106] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:30:36 2022] POST /identity/v3/projects => generated 302 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:37.440975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:37.489221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ff67a-41df-48f9-b21c-7705145bdef1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:37.584889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:37.585746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:37.586167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:37.586601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:37.587409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:37.588430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:37.590110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:37.608384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 48/107] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:37.616475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:37.704443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efbfcf-0bea-452e-b123-8e25b761825a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:37.756556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:37.757156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:37.757682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:37.758407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:37.773957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:37.779580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:37.787233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:37.788244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:37.789086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:37.791843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 60/108] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:37.799293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:37.840965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56193cec-8d43-4ef1-95d8-e61b5d8d5a42 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:37.877993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:37.904581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:37.905244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:37.905683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:37.906139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:37.912257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:37.913961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:37.916196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 49/109] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:37 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:37.925712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:37.926435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86bd52-13b6-4fae-9e7a-4fd67758d265 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:37.968220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:37.970843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:37.971222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:37.971600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:37.972419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:37.972804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:37.974312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:37.985099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 61/110] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:37.991459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:38.063136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:38.111114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee88bb-2a47-4bdc-9ca7-867a05398b15 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:38.150329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaa920-333e-4f5b-811a-6abac2f342a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:38.151209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaa920-333e-4f5b-811a-6abac2f342a8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.151521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaa920-333e-4f5b-811a-6abac2f342a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.151807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaa920-333e-4f5b-811a-6abac2f342a8 None admin] PATH_INFO: `/v3/users/94fc7cef8245497295728761ac39788e` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.190229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaa920-333e-4f5b-811a-6abac2f342a8 None admin] RBAC: Authorizing `identity:get_user(user_id=94fc7cef8245497295728761ac39788e)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:38.192489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaa920-333e-4f5b-811a-6abac2f342a8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:38.194516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 62/111] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:37 2022] GET /identity/v3/users/94fc7cef8245497295728761ac39788e => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:38.196098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a182b-b886-4d6c-b012-ae517a4c9137 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.196805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a182b-b886-4d6c-b012-ae517a4c9137 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.197824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a182b-b886-4d6c-b012-ae517a4c9137 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.199625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 63/112] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:38.205270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.205809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.206340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.234332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:38.235894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:38.273729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b004-ce9e-4a41-b292-3afe66a7fbb6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:38.353832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:38.354833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.355313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.355814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.357220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:38.358879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:38.360759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 50/113] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:37 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:38.361846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:38.370317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 64/114] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:38.371461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:38.472504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:38.552917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f4c55-d939-437f-bd28-a26f9ea5e08c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:38.585300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:38.652858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d3a5c-11a3-4bb9-920d-adebb7001ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:38.653421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d3a5c-11a3-4bb9-920d-adebb7001ccf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.653703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d3a5c-11a3-4bb9-920d-adebb7001ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.653975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d3a5c-11a3-4bb9-920d-adebb7001ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.763775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d3a5c-11a3-4bb9-920d-adebb7001ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:38.765424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d3a5c-11a3-4bb9-920d-adebb7001ccf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:38.781296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09706afe-fe5a-4ac7-ad09-92f9d4cf3655 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:38.830936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:38.835637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717c1db-abc0-416e-9b1f-85cda92aa2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:38.836725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717c1db-abc0-416e-9b1f-85cda92aa2f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.837140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717c1db-abc0-416e-9b1f-85cda92aa2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.837567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717c1db-abc0-416e-9b1f-85cda92aa2f2 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/ce85ea6bdfa54b3298ecf834a6d7fd47/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.840419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717c1db-abc0-416e-9b1f-85cda92aa2f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b2ffbd61a140daaf31e00fa35ab0b2, user_id=ce85ea6bdfa54b3298ecf834a6d7fd47, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:38.842195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717c1db-abc0-416e-9b1f-85cda92aa2f2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:38.858498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 65/115] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:38 2022] PUT /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/ce85ea6bdfa54b3298ecf834a6d7fd47/roles/87f1831257824560813f954348534540 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:38.860351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309bdce-ef14-4d3b-9d4c-03023fbcbfc0 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.860850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309bdce-ef14-4d3b-9d4c-03023fbcbfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.861312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309bdce-ef14-4d3b-9d4c-03023fbcbfc0 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.862972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 66/116] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:38.864498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec917b39-dd25-4a1a-b29b-4e40a33feab9 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.864988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec917b39-dd25-4a1a-b29b-4e40a33feab9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.865472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec917b39-dd25-4a1a-b29b-4e40a33feab9 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.871982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 67/117] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:38.873274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec917b39-dd25-4a1a-b29b-4e40a33feab9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:38.887098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 51/118] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:38.888398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5134b91-8831-41cd-9742-e977034cf0be None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:38.888735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5134b91-8831-41cd-9742-e977034cf0be None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:38.889015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5134b91-8831-41cd-9742-e977034cf0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:38.915729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c5134b91-8831-41cd-9742-e977034cf0be None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:38.922753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c5134b91-8831-41cd-9742-e977034cf0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:39.039251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec917b39-dd25-4a1a-b29b-4e40a33feab9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:39.053153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 52/119] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:39.061127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:39.061435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:39.061688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:39.067905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:39.122620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:39.123728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:39.124220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:39.124724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:39.127088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:39.128353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:39.145387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:39.151216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:39.152077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:39.152844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:39.155880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 68/120] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:39.157449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:39.295726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 53/121] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:39.297416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:39.352939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:39.414113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c12e-aba7-46c8-89ab-fdbd09c2d387 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:39.511517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210baab-6936-480b-aee1-06af477de405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:39.514262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210baab-6936-480b-aee1-06af477de405 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:39.514547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210baab-6936-480b-aee1-06af477de405 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:39.514823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210baab-6936-480b-aee1-06af477de405 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:39.523557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210baab-6936-480b-aee1-06af477de405 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:39.523557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210baab-6936-480b-aee1-06af477de405 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:39.523557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 69/122] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:39 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:39.536075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210baab-6936-480b-aee1-06af477de405 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:39.636572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:39.663505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb260218-e5d1-4836-ac88-94132ca03be7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:39.770513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:39.771514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:39.773660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:39.774134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:39.776130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:39.776655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:39.778492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:39.799418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 54/123] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 504 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:39.800231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:39.817513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210baab-6936-480b-aee1-06af477de405 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:39.867501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:39.868509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:39.869501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:39.870345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:39.888739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:39.894618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:39.895827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:39.896596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:39.898923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 70/124] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:39.905256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:39.961741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:39.997249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45d47d-df20-4b3b-9beb-bc342cfbcb47 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.026152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:40.034894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dd660-72ba-46a2-bf28-5985f0ee23c6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.052514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:40.053142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.053771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.054849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] RBAC: Adding query filter params (role.id=9a827b3ce4b243f494208e3f97029a8d, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=94fc7cef8245497295728761ac39788e) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:40.055159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:40.058539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:40.069082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 55/125] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:39 2022] GET /identity/v3/role_assignments?user.id=94fc7cef8245497295728761ac39788e&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=9a827b3ce4b243f494208e3f97029a8d => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:40.069944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:40.084097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:40.085106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.086306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.086783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.105759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:40.114547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:40.115530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:40.116314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:40.116893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 71/126] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:30:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:40.118835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:40.267798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88935-aaee-4068-9f6d-2da1baef4c92 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.320485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:40.322174 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.322670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.323155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.327513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:40.329358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:40.335631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 56/127] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:40 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:40.336909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:40.361338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e0849-5f37-4df0-b1cc-fa243f873452 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.377735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:40.417006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:40.418413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.419851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.420355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.421275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:40.421781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:40.423499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:40.439406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 72/128] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:40.440602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:40.507030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:40.651637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78dbe-a0b4-407f-ac48-50d9ac2c83b0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.687311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6c9c-17d6-4e24-867d-b471794d22fa None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.729991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:40.731319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.732406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.732898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.734256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:40.736456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:40.738770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 73/129] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:40 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:40.739835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:40.741811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:40.742364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.742652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.742929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.751524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:40.751813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:40.753251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef74945-9ec1-4a45-b9d6-8eff6d80c9a0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:40.766538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 57/130] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:30:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:40.768414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849d800-d2b2-45b5-853e-a57cb29f468b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:40.768917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849d800-d2b2-45b5-853e-a57cb29f468b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:40.769398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849d800-d2b2-45b5-853e-a57cb29f468b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:40.770941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 58/131] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:40.772839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849d800-d2b2-45b5-853e-a57cb29f468b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:40.888015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:40.927137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cafc42-e311-4929-8268-b53854cc7a8b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:40.945479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:40.992541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849d800-d2b2-45b5-853e-a57cb29f468b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:41.009931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:41.011045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.011602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.012072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.013709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:41.016530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:41.019444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 74/132] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:40 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:41.022790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:41.059280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b33bf-3da0-4489-9c76-ee4b70034c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:41.060556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b33bf-3da0-4489-9c76-ee4b70034c09 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.061061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b33bf-3da0-4489-9c76-ee4b70034c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.061609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b33bf-3da0-4489-9c76-ee4b70034c09 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/34b8b29876a54a4285fc37bd5edb4042/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.063934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b33bf-3da0-4489-9c76-ee4b70034c09 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b2ffbd61a140daaf31e00fa35ab0b2, user_id=34b8b29876a54a4285fc37bd5edb4042, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:41.065825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b33bf-3da0-4489-9c76-ee4b70034c09 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:41.093482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 59/133] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:40 2022] PUT /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/34b8b29876a54a4285fc37bd5edb4042/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:41.098313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.100358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.100927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.143919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:41.145623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:41.246517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58e60e-9b67-465d-8544-f9a0e3b68cc6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:41.303504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:41.305236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.305539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.307513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.308194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:41.311648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:41.315059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 75/134] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:41 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:41.318280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 60/135] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:41.320441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:41.333307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:41.371534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:41.450975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:41.466064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96526fa8-5f64-4943-b6b8-316b8faf9a48 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:41.518994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352a78-b95f-4ad3-b471-11a56f421ccf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:41.536226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c97c0-8f61-4b9a-874b-87fc4fa2959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:41.536875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c97c0-8f61-4b9a-874b-87fc4fa2959a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.537157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c97c0-8f61-4b9a-874b-87fc4fa2959a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.537429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c97c0-8f61-4b9a-874b-87fc4fa2959a None admin] PATH_INFO: `/v3/projects/1b1bd105fca54dde86786cbabad26012` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.538635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c97c0-8f61-4b9a-874b-87fc4fa2959a None admin] RBAC: Authorizing `identity:get_project(project_id=1b1bd105fca54dde86786cbabad26012)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:41.540188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c97c0-8f61-4b9a-874b-87fc4fa2959a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:41.542584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 76/136] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:41 2022] GET /identity/v3/projects/1b1bd105fca54dde86786cbabad26012 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:41.548301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9c31-23ee-4120-ad62-bccc21d7a92e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.548588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9c31-23ee-4120-ad62-bccc21d7a92e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.551352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9c31-23ee-4120-ad62-bccc21d7a92e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.551352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 77/137] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:41.551352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9c31-23ee-4120-ad62-bccc21d7a92e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:41.593251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb743a-ba87-4c91-803b-2610c1542588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:41.594163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb743a-ba87-4c91-803b-2610c1542588 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.594588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb743a-ba87-4c91-803b-2610c1542588 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.594999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb743a-ba87-4c91-803b-2610c1542588 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.596356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb743a-ba87-4c91-803b-2610c1542588 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:41.597965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb743a-ba87-4c91-803b-2610c1542588 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:41.599767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 61/138] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:41 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:41.601027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.603448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.604186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.645581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:41.647021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:41.727739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:41.743638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9c31-23ee-4120-ad62-bccc21d7a92e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:41.798628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:41.799287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 62/139] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:41.799784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:41.800844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:41.801262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] PATH_INFO: `/v3/projects/1b1bd105fca54dde86786cbabad26012/users/ce85ea6bdfa54b3298ecf834a6d7fd47/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:41.804138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b1bd105fca54dde86786cbabad26012, user_id=ce85ea6bdfa54b3298ecf834a6d7fd47, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:41.804345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:41.805951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:41.830811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 78/140] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:41 2022] PUT /identity/v3/projects/1b1bd105fca54dde86786cbabad26012/users/ce85ea6bdfa54b3298ecf834a6d7fd47/roles/87f1831257824560813f954348534540 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:41.832069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:41.986102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcb200-15f6-4cb8-8ac1-8ae388ec5830 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:41.986750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:42.046077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:42.047080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:42.047604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:42.048059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:42.051755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:42.053467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:42.055375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 79/141] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:42.064515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:42.101130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:42.106041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f22a-323b-463b-b8e4-432cf62ac4e1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:42.219507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:42.219507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:42.219507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:42.219507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/ce85ea6bdfa54b3298ecf834a6d7fd47/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:42.227889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b2ffbd61a140daaf31e00fa35ab0b2, user_id=ce85ea6bdfa54b3298ecf834a6d7fd47, role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:42.227889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:42.267565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 63/142] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:41 2022] PUT /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/ce85ea6bdfa54b3298ecf834a6d7fd47/roles/1d2b7e763c90438eab518a55d7589260 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:42.268919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:42.274086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606757aa-da41-47ba-ad83-b860106fcada None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:42.331354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:42.332672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:42.333163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:42.333641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:42.334527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:42.338247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:42.372285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 80/143] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:30:42 2022] POST /identity/v3/users => generated 283 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:42.653927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c71a-5755-4b7c-9755-41389ea29ab1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:42.725481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:42.780991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:42.781665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:42.781954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:42.787515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:42.821325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:42.827133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:42.828110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:42.828898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:42.836890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 64/144] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 568 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:42.845011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:43.001373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:43.042356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa33e4-6319-4949-a226-40f89b4d6fb5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:43.112238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:43.113238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:43.113723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:43.114224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:43.115113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:43.115643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:43.117234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b8a5-c7d9-411f-9183-054e4b010397 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:43.126502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 81/145] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:43.132462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:43.143400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a3b5a-06d2-4921-b5fd-2140805aab0a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:43.144309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a3b5a-06d2-4921-b5fd-2140805aab0a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:43.144799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a3b5a-06d2-4921-b5fd-2140805aab0a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:43.147417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 82/146] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:43.153618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072163-027b-472f-8860-ce22451949d6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:43.154772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072163-027b-472f-8860-ce22451949d6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:43.155250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072163-027b-472f-8860-ce22451949d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:43.196311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ba072163-027b-472f-8860-ce22451949d6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:43.197876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ba072163-027b-472f-8860-ce22451949d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:43.372871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca0dac-5c4c-47d8-879d-f8820462df0b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:43.388297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 83/147] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:43.432523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:43.433508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:43.434016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:43.434501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:43.439043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:43.440782 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:43.442664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 65/148] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:43 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:43.452615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072163-027b-472f-8860-ce22451949d6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:43.567514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:43.608082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072163-027b-472f-8860-ce22451949d6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:43.658340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:43.658886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:43.659157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:43.659417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:43.752671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:43.754719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:43.774192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:43.825563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:43.866739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 84/149] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:43.882678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:43.920346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:44.014037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542cfc9-689f-4fcb-8ef7-0f737e0d7101 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:44.066988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe242e4-435f-4916-b3c9-064ad5401ef4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:44.088807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:44.089830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.090307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.090786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.124714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:44.132262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:44.132911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.133428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.133704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.136270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:44.136969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:44.137794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:44.138632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:44.139211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:44.139806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 85/150] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:43 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:44.145907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 66/151] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:30:44.146036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:44.147112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:44.207393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:44.329134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261239d-4989-439b-a921-7f1f8ac084a5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:44.355386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6717-151c-4b47-9e76-524d0dcf8691 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:44.382666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:44.413269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:44.413911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.414226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.414439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.415181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=ce85ea6bdfa54b3298ecf834a6d7fd47) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:44.415535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:44.417812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:44.430726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 86/152] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:44 2022] GET /identity/v3/role_assignments?user.id=ce85ea6bdfa54b3298ecf834a6d7fd47&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=87f1831257824560813f954348534540 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:44.436277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:44.437232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.437731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.438220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.439151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:44.439730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:44.441381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44057298-9a42-4572-bd91-c08e4eaa9640 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:44.456069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 67/153] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:44.466385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:44.742106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:44.746847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7212bb-b36a-45d8-800b-7277195832ff None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:44.839219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d9c9-e2af-468f-a36b-41128b671855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:44.840510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d9c9-e2af-468f-a36b-41128b671855 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.841299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d9c9-e2af-468f-a36b-41128b671855 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.842034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d9c9-e2af-468f-a36b-41128b671855 None admin] PATH_INFO: `/v3/users/94fc7cef8245497295728761ac39788e` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.843222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d9c9-e2af-468f-a36b-41128b671855 None admin] RBAC: Authorizing `identity:get_user(user_id=94fc7cef8245497295728761ac39788e)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:44.845915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d9c9-e2af-468f-a36b-41128b671855 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:44.852542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 87/154] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:44 2022] GET /identity/v3/users/94fc7cef8245497295728761ac39788e => generated 289 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:44.853376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01609e-30be-49e3-bad6-9d6692ec286b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.854183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01609e-30be-49e3-bad6-9d6692ec286b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.854651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01609e-30be-49e3-bad6-9d6692ec286b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.856218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 68/155] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:44.867505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:44.867505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:44.867505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:44.868091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01609e-30be-49e3-bad6-9d6692ec286b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:44.937668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:44.943513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:45.202352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 88/156] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:45.240189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01609e-30be-49e3-bad6-9d6692ec286b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:45.312193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:45.313218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:45.313705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:45.314194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:45.315571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:45.317387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:45.319244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 69/157] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:44 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:45.335639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:45.532264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f5139-f787-4d53-80aa-2b452a93b379 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:45.539792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:45.585111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:45.585773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:45.591520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:45.591520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/94fc7cef8245497295728761ac39788e/roles/9a827b3ce4b243f494208e3f97029a8d` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:45.594821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] RBAC: Authorizing `identity:create_grant(project_id=74b2ffbd61a140daaf31e00fa35ab0b2, user_id=94fc7cef8245497295728761ac39788e, role_id=9a827b3ce4b243f494208e3f97029a8d)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:45.596474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:45.632010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 89/158] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:45 2022] PUT /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/users/94fc7cef8245497295728761ac39788e/roles/9a827b3ce4b243f494208e3f97029a8d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:45.867387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:45.919256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:45.994542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257125-fa20-413e-b7c8-dced50d5ad45 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:46.039216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:46.039940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.040222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.040501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.057331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:46.062784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:46.063652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:46.064496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 70/159] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:46.069873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:46.213668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca416ba4-3797-4d8b-b812-6b38a9c929bd None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:46.269521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:46.270189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.270472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.270743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.271459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:46.271785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:46.273273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:46.287684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 90/160] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:46.294503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:46.408883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a738c-e2fa-4b41-9d11-e3a11427d5c4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:46.410422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:46.453489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c950c3f-e449-4d68-97fd-24801de94e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:46.454127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950c3f-e449-4d68-97fd-24801de94e56 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.454402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950c3f-e449-4d68-97fd-24801de94e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.454671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950c3f-e449-4d68-97fd-24801de94e56 None admin] PATH_INFO: `/v3/users/34b8b29876a54a4285fc37bd5edb4042` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.455846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c950c3f-e449-4d68-97fd-24801de94e56 None admin] RBAC: Authorizing `identity:get_user(user_id=34b8b29876a54a4285fc37bd5edb4042)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:46.457299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c950c3f-e449-4d68-97fd-24801de94e56 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:46.458889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 71/161] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:46 2022] GET /identity/v3/users/34b8b29876a54a4285fc37bd5edb4042 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:46.465443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:46.501225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:46.536830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22dc5-5bdf-4b83-893a-7b23b668717e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.537494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22dc5-5bdf-4b83-893a-7b23b668717e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.537965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22dc5-5bdf-4b83-893a-7b23b668717e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.542015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 72/162] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:46.547257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564907f5-f74f-4e8f-b5f8-e30edb439e0a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.547821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564907f5-f74f-4e8f-b5f8-e30edb439e0a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.548289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564907f5-f74f-4e8f-b5f8-e30edb439e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.579625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-564907f5-f74f-4e8f-b5f8-e30edb439e0a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:46.581269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-564907f5-f74f-4e8f-b5f8-e30edb439e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:46.615506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772877-96fd-4d06-b16d-d9ec61906036 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:46.667507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:46.667507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.667507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.667507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.700255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 73/163] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:46.741916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059b12-2b8a-4a48-85fa-233e8b0f801a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.742185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059b12-2b8a-4a48-85fa-233e8b0f801a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.747610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059b12-2b8a-4a48-85fa-233e8b0f801a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.747610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 74/164] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:46.757609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:46.757609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:46.757609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:46.774495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:46.776269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:46.791268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:46.796680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:46.806303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:46.933526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 75/165] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:47.051170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 91/166] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:47.060713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:47.114625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:47.203937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:47.219504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33240c0c-949d-4023-89b6-d3bb947895bd None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:47.335501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ab26f-8e0e-424f-8615-2d5f9d12abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:47.335501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ab26f-8e0e-424f-8615-2d5f9d12abeb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.335501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ab26f-8e0e-424f-8615-2d5f9d12abeb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.335501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ab26f-8e0e-424f-8615-2d5f9d12abeb None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.335894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ab26f-8e0e-424f-8615-2d5f9d12abeb None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:47.335894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ab26f-8e0e-424f-8615-2d5f9d12abeb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:47.335894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 76/167] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:47.335894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee140d2-6c38-426b-b40e-6bef17fdbaf0 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.343618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee140d2-6c38-426b-b40e-6bef17fdbaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.343948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee140d2-6c38-426b-b40e-6bef17fdbaf0 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.345527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 77/168] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:47.346251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee140d2-6c38-426b-b40e-6bef17fdbaf0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:47.464593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:47.468053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee140d2-6c38-426b-b40e-6bef17fdbaf0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:47.505704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:47.506548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.507294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.507730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.509037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=34b8b29876a54a4285fc37bd5edb4042) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:47.509640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:47.511393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b07b1a-fe0c-4e6e-8740-425ec9be8e49 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:47.517914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 78/169] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/role_assignments?user.id=34b8b29876a54a4285fc37bd5edb4042&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:47.519544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.520295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.520755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.522511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd5d02-e8c3-49db-a31f-c4c6340b3068 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:47.542328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:47.543909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:47.592024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:47.624973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:47.625631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.625911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.626187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.637782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:47.647889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 79/170] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:47.648563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:47.652852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:47.653764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:47.655084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 92/171] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:47.660494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:47.850691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e96e1c-cae5-40fa-b776-bb4c0fb24a6c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:47.860498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:47.886056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6836f3-6c70-4016-b670-3efcbdc2c360 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:47.903553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:47.904177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.904457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.904726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.905404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:47.905707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:47.907087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:47.920633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 93/172] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:47.929192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:47.951534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:47.952173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:47.952607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:47.953010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:47.984350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:47.989657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:47.990773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:47.991756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:47.992670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:47.994224 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 80/173] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:48.000873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.103997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217480a4-15eb-41a5-9305-f738918da163 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.169710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.170806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.172497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.172953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.175630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:48.178163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:48.180348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 94/174] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:48.182933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.188419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a9f8a-b4f8-4c84-b7ce-dd0ec165f47c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.256384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.258572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.259023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.259984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.260808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:48.261245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:48.263404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:48.276956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 81/175] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:30:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:48.278388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.299427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:48.368595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98093e-8245-4431-a37e-d258bbaeaa12 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.379834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:48.400622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da280604-10bd-4b05-bbfb-b5982ce4866f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.408095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.408956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.409524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.409981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.421231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:48.427129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:48.428144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:48.429112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:48.431976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 95/176] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:48.432871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.437916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.438528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.438805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.439075 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.511982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:48.513789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:48.530041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:48.538738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1591e-a2df-4a83-ad0a-fde91316d04b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.581121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b055516-1e95-4e91-b542-06a2627b4874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.581763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b055516-1e95-4e91-b542-06a2627b4874 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.582168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b055516-1e95-4e91-b542-06a2627b4874 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.582298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b055516-1e95-4e91-b542-06a2627b4874 None admin] PATH_INFO: `/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.583458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b055516-1e95-4e91-b542-06a2627b4874 None admin] RBAC: Authorizing `identity:get_user(user_id=ce85ea6bdfa54b3298ecf834a6d7fd47)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:48.584959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b055516-1e95-4e91-b542-06a2627b4874 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:48.586694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 96/177] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/users/ce85ea6bdfa54b3298ecf834a6d7fd47 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:48.589614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa7a65-2a32-4f58-a6e2-93d1190775d9 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.590061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa7a65-2a32-4f58-a6e2-93d1190775d9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.590708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa7a65-2a32-4f58-a6e2-93d1190775d9 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.593025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 97/178] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:48.594217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9a0f8-00da-4789-8897-a3fcc63ef5ad None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.594635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9a0f8-00da-4789-8897-a3fcc63ef5ad None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.595055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9a0f8-00da-4789-8897-a3fcc63ef5ad None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.596790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 98/179] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:48.597686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9a0f8-00da-4789-8897-a3fcc63ef5ad None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.632825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 82/180] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:48.632825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.642779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:48.712543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9a0f8-00da-4789-8897-a3fcc63ef5ad None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.744568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453e82-6d4b-44ac-8be2-b5a23a894d24 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:48.747283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.747923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.748196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.748461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.749120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:48.749416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:48.750768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0d561-6ee4-4c52-9830-bb98e32f09f3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:48.760429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 99/181] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:48.761547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e341a8-a111-4148-9ca4-936e3cbf5306 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.761831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e341a8-a111-4148-9ca4-936e3cbf5306 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.762097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e341a8-a111-4148-9ca4-936e3cbf5306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.764150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:48.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-31e341a8-a111-4148-9ca4-936e3cbf5306 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:48.781436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-31e341a8-a111-4148-9ca4-936e3cbf5306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:48.787345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:48.787981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.788259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.788527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.855652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 100/182] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:48.857004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:48.857293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:48.857566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:48.861413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:48.862889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:48.885836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:48.886991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:48.911953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:48.966294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 101/183] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:48.972426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:48.996941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 83/184] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:48.998767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.017223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.121461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb7673-7e7d-423b-80e1-0028f251dfc8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.162343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.162998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.163284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.163586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] PATH_INFO: `/v3/users/e5d547f1548f40e18a9723578f675afd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.167277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b38d9d-8cb6-400c-bbaa-edab6d21913a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.178079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=e5d547f1548f40e18a9723578f675afd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.179728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.181772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 84/185] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/users/e5d547f1548f40e18a9723578f675afd => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.182875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.204175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.204806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.205093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.205341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] PATH_INFO: `/v3/projects/1b1bd105fca54dde86786cbabad26012` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.206707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] RBAC: Authorizing `identity:get_project(project_id=1b1bd105fca54dde86786cbabad26012)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.208170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.209780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 102/186] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:48 2022] GET /identity/v3/projects/1b1bd105fca54dde86786cbabad26012 => generated 312 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.210540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.238624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.265966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.304049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574358fb-5890-47bb-85b7-9920fda92fa2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.319681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedee14-73f8-4e85-8e6b-0ce39cb35dec None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.345640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.346272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.346551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.346820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.347969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.349444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.351066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 85/187] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.351850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.355785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18518be5-acb5-47eb-ad91-2c6c3efec0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.356393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518be5-acb5-47eb-ad91-2c6c3efec0bf None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.356675 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518be5-acb5-47eb-ad91-2c6c3efec0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.356946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518be5-acb5-47eb-ad91-2c6c3efec0bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.367745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18518be5-acb5-47eb-ad91-2c6c3efec0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-18518be5-acb5-47eb-ad91-2c6c3efec0bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:49.373152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:49.374564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:49.375345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 103/188] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:49.376039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9a40c-ee93-4e5f-b896-292e42045c5b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.376104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9a40c-ee93-4e5f-b896-292e42045c5b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.376597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9a40c-ee93-4e5f-b896-292e42045c5b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.377726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 104/189] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:49.378338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9a40c-ee93-4e5f-b896-292e42045c5b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.454417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dc195-2125-4bf9-a2fd-34ea00869ee6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.464868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.481892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9a40c-ee93-4e5f-b896-292e42045c5b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.487573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.488202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.488476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.488741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.499282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.515975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.516560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.516830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.517102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.517818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, scope.project.id=1b1bd105fca54dde86786cbabad26012, user.id=ce85ea6bdfa54b3298ecf834a6d7fd47) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:49.518142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.519575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.525158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 105/190] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/role_assignments?user.id=ce85ea6bdfa54b3298ecf834a6d7fd47&scope.project.id=1b1bd105fca54dde86786cbabad26012&role.id=87f1831257824560813f954348534540 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.525926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.558545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.560097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.675151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c660a0-7637-4f2e-adb2-b4a773b6c7a0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.685634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 86/191] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:49.686731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.720925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.721491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.721771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.722043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.723159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.724656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.726230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 106/192] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.727093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.735840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.748830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:49.801944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051286e-16b6-467e-80b1-819e87670afb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.831724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09d40-960b-4f76-9089-1d0ce5ddde4b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:49.835818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.836414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.836689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.836958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.838252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] RBAC: Adding query filter params (role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=ce85ea6bdfa54b3298ecf834a6d7fd47) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:49.838555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.840034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a39a6-1de3-49f4-b925-6879dc9abc88 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.845821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 87/193] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/role_assignments?user.id=ce85ea6bdfa54b3298ecf834a6d7fd47&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=1d2b7e763c90438eab518a55d7589260 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.846757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.847033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.847299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.873250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:49.874495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:49.874753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:49.875380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:49.875688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:49.875959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:49.876650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:49.876951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:49.878360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:49.888578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 107/194] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:49.894612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:49.953567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 88/195] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:49.954451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.002848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.018228 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.054045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29eeccb-6811-4f7f-8224-3190eb19bac0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.084306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f6cfd-ae58-4d3f-ad28-d8c5b3230aab None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.096246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.096880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.097450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.097733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.098907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.100503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.102169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 108/196] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:49 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:50.103856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.126956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.127661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.127939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.128211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.128855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.130291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.157830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 89/197] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:30:49 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:50.158787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.246507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051c98b-8938-443c-84ce-d81b28678ab0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.266601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.281366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8221d-2a1c-441f-9e9f-b7df47d77669 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.286487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.287155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.287461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.287761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] PATH_INFO: `/v3/users/e5d547f1548f40e18a9723578f675afd` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.289140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] RBAC: Authorizing `identity:get_user(user_id=e5d547f1548f40e18a9723578f675afd)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.291255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.293252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 109/198] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/users/e5d547f1548f40e18a9723578f675afd => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:50.299188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.320425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.323364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.324380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.324657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.324924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.326189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=e5d547f1548f40e18a9723578f675afd) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:50.326189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.327438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.334008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 90/199] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/role_assignments?user.id=e5d547f1548f40e18a9723578f675afd&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:50.334650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.432672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc50343-c20a-441d-b564-0f058482f1bf None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.482725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.483303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b8304-152b-4857-b0f5-5f061c18b66e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.483410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.483750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.484027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.498292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:50.504732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:50.505773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:50.506687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 110/200] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:50.512783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.529596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.530131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.530410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.530681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.561170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.627869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.628448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.630529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.653107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c9778-0c56-4d3e-9460-0a79638c2354 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.705259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.706008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.706384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.706750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.707867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:50.708303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.709767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.722382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 111/201] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:50.727814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.792079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 91/202] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:50.797932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:50.902731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:50.933851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7918-720b-4547-af18-3fb6224f4adc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.975812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:50.976734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:50.977399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:50.977869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] PATH_INFO: `/v3/users/94fc7cef8245497295728761ac39788e` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:50.980752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] RBAC: Authorizing `identity:get_user(user_id=94fc7cef8245497295728761ac39788e)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:50.982589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:50.984658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60560-059a-41f0-a94f-3484b0daec45 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:50.986438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 112/203] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/users/94fc7cef8245497295728761ac39788e => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:50.991965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:51.004362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:51.030149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:51.031929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:51.032386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:51.032910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:51.034208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:51.035879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:51.037844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 92/204] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:51.044745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:51.180407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21d099-5677-4b86-b5f3-3b3ef2d95303 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:51.198583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092b566-245a-4252-b6e6-422e7502f478 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:51.216035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:51.216405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:51.216692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:51.216966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:51.242449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:51.271871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:51.272953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:51.273401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:51.273845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:51.275076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] RBAC: Adding query filter params (role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=e5d547f1548f40e18a9723578f675afd) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:51.280128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:51.281869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:51.287627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:51.289122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:51.292390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 93/205] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:51 2022] GET /identity/v3/role_assignments?user.id=e5d547f1548f40e18a9723578f675afd&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=1d2b7e763c90438eab518a55d7589260 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:51.302919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:51.402175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 113/206] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 00:30:51.408712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:51.554584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db928fd-d018-4e9f-b98b-ac9af80e7fd0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:51.596876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9547ad-fcf2-40d5-a0a5-eb7d2ffeff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:51.597849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9547ad-fcf2-40d5-a0a5-eb7d2ffeff57 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:51.598319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9547ad-fcf2-40d5-a0a5-eb7d2ffeff57 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:51.598776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9547ad-fcf2-40d5-a0a5-eb7d2ffeff57 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:51.600186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9547ad-fcf2-40d5-a0a5-eb7d2ffeff57 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:51.601839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9547ad-fcf2-40d5-a0a5-eb7d2ffeff57 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:51.608101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 94/207] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:51 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:51.614174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:51.747068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40114344-fe70-462e-be26-2d3ec229efc2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:51.754558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:51.792413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:51.792822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:51.793107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:51.793378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:51.794109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] RBAC: Adding query filter params (role.id=9a827b3ce4b243f494208e3f97029a8d, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2, user.id=94fc7cef8245497295728761ac39788e) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:51.794416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:51.795892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762e082-156e-468e-b041-872f4c8d1884 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:51.802056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 114/208] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:51 2022] GET /identity/v3/role_assignments?user.id=94fc7cef8245497295728761ac39788e&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=9a827b3ce4b243f494208e3f97029a8d => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:52.803144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770d7ca-7069-4a0c-92e1-96eed3deeb1f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:52.803674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770d7ca-7069-4a0c-92e1-96eed3deeb1f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:52.803778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770d7ca-7069-4a0c-92e1-96eed3deeb1f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:52.805186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 95/209] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:52.810434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cbd93-4d29-4bb9-97da-e74299eec7bc None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:52.810902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cbd93-4d29-4bb9-97da-e74299eec7bc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:52.810979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cbd93-4d29-4bb9-97da-e74299eec7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:52.821752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe03b-6e49-4145-9dda-31c85695fcb7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:52.822565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe03b-6e49-4145-9dda-31c85695fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:52.822827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe03b-6e49-4145-9dda-31c85695fcb7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:52.824193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 96/210] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:52.829385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871346-a171-4a40-8fa2-dd97df49c1d9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:52.829665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871346-a171-4a40-8fa2-dd97df49c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:52.829940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871346-a171-4a40-8fa2-dd97df49c1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:52.830698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-f37cbd93-4d29-4bb9-97da-e74299eec7bc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:52.831925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f37cbd93-4d29-4bb9-97da-e74299eec7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:52.849468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-59871346-a171-4a40-8fa2-dd97df49c1d9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:52.850898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-59871346-a171-4a40-8fa2-dd97df49c1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:52.913805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:52.919393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 115/211] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:52.927079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 97/212] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:52.988839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b4967-27f5-4766-aa93-efcb558f5198 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:52.989136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b4967-27f5-4766-aa93-efcb558f5198 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:52.989398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b4967-27f5-4766-aa93-efcb558f5198 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:52.990750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 116/213] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:52.995983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:52.996331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:52.996606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.015268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:53.016498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:53.082230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 98/214] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:53.189359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b4967-27f5-4766-aa93-efcb558f5198 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.204555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.295855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:53.300620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b4967-27f5-4766-aa93-efcb558f5198 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.341510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.341710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e886b-e5c8-4d76-9410-9598f5f054d4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.342628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.343099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.343602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.355330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:53.358486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:53.360264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:53.361390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:53.362201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 117/215] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:53.362842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.391547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.392177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.392600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.392875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.406002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:53.412661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:53.413516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:53.414339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 99/216] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:30:53.414986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.471817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd86faa-4a28-4fc8-96f2-239c8675e91a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.509000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.510152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.510785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.510785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.511852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:53.513292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:53.519424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 118/217] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:53.519424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.524733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d32a9f-78a9-445b-8a28-161d97134ef5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.547890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:53.563182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.564096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.564589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.565069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.565959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:53.566429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:53.568027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:53.577146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 100/218] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:53.578282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.617992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:53.626239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ec8bd-f445-4ce6-909c-7e42807122e5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.663053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.663694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.663967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.664232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.665301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:53.665724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:53.669403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:53.679932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 119/219] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:53.685882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.698027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0f136-877c-42b6-a31f-9b73f8110890 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.735044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.735598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.735871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.736136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.736773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:53.738211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:53.761708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 101/220] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:30:53 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:30:53.770976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.789972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:53.804299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aeb02-c23f-49cf-88a0-1c9f6f74a950 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.845662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d621e26-8dff-4446-86ff-a45cb4aebb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.846409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621e26-8dff-4446-86ff-a45cb4aebb62 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.846761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621e26-8dff-4446-86ff-a45cb4aebb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.847032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621e26-8dff-4446-86ff-a45cb4aebb62 None admin] PATH_INFO: `/v3/users/e5d547f1548f40e18a9723578f675afd` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.848262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d621e26-8dff-4446-86ff-a45cb4aebb62 None admin] RBAC: Authorizing `identity:get_user(user_id=e5d547f1548f40e18a9723578f675afd)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:53.849873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d621e26-8dff-4446-86ff-a45cb4aebb62 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:53.851745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 120/221] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/users/e5d547f1548f40e18a9723578f675afd => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:53.852903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4adbb2-d555-4247-9419-162fa6d45b3b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.853802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4adbb2-d555-4247-9419-162fa6d45b3b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.854076 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4adbb2-d555-4247-9419-162fa6d45b3b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.859595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 121/222] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:53.859595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4adbb2-d555-4247-9419-162fa6d45b3b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:53.901125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:53.911926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3917d9c-f2e3-4ebd-a8b0-dea2ba21938b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.949109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b255d-ef3f-46d5-bbc0-6d6ff949c1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:53.949740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b255d-ef3f-46d5-bbc0-6d6ff949c1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.950011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b255d-ef3f-46d5-bbc0-6d6ff949c1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.950328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b255d-ef3f-46d5-bbc0-6d6ff949c1c0 None admin] PATH_INFO: `/v3/users/d78b599ca2294fbb81c65eb58931ca02` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.967727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b255d-ef3f-46d5-bbc0-6d6ff949c1c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d78b599ca2294fbb81c65eb58931ca02)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:53.969216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b255d-ef3f-46d5-bbc0-6d6ff949c1c0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:53.971841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 102/223] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/users/d78b599ca2294fbb81c65eb58931ca02 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:53.972588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:53.972588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:53.972757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:53.974090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4adbb2-d555-4247-9419-162fa6d45b3b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:53.992869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:53.994055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:54.015937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.016577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.016837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.017225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.018326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:54.019898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:54.021690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 122/224] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:53 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:54.022534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.056379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:54.075936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 103/225] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:54.076896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.148900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded869a4-2c76-49da-890d-e9190841041f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:54.173488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:54.185077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.186027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.186512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.187022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.197418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bcdfb-f0b8-4a44-b603-e48623e6682b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:54.236082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.237159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.237772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.238336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/e5d547f1548f40e18a9723578f675afd/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.240504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] RBAC: Authorizing `identity:create_grant(project_id=0718a24693104c6eada4aabe8e2ba6b6, user_id=e5d547f1548f40e18a9723578f675afd, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:54.242376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:54.261002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:54.262464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:54.264320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 104/226] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:54 2022] PUT /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/e5d547f1548f40e18a9723578f675afd/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:54.295696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:54.380437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 123/227] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:54.386296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.479922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.500843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3aaa-3597-41ca-9870-b98eb541cc02 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:54.551337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.552005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.552344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.552680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.554670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:54.556733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:54.559050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 105/228] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:54 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:54.564882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.580634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:54.611217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de5cb3-2a7f-49b4-9ff1-77f55f255630 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:54.647886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:54.653084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.654019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.654571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.654957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.668906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:54.674243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:54.675325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:54.676226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:54.677988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 124/229] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:30:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:30:54.684402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.767302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3b916-60ab-41de-b4b0-53643b05f595 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:54.831506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39232617-3388-4e73-bf1f-e5eef938bbee None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:54.834812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.835795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.836268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.836729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.837693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=d78b599ca2294fbb81c65eb58931ca02) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:54.838265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:54.840218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:54.858960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 106/230] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:54 2022] GET /identity/v3/role_assignments?user.id=d78b599ca2294fbb81c65eb58931ca02&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=87f1831257824560813f954348534540 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:54.861285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:54.861982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:54.862336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:54.862681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:54.863436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:54.863840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:54.865356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:54.875182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 125/231] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:30:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:54.881117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:54.958390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:54.993337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f3463-717d-441c-9248-e0dce8b4f4c8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:55.030577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:55.031615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:55.032120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:55.034277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] PATH_INFO: `/v3/users/e5d547f1548f40e18a9723578f675afd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:55.035871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=e5d547f1548f40e18a9723578f675afd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:55.037574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:55.039708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 107/232] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:54 2022] GET /identity/v3/users/e5d547f1548f40e18a9723578f675afd => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:55.048655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:55.123622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:55.194289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a46bf-12b7-4899-84a3-79d7ca01e2e8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:55.245218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91878cf7-0a65-43bd-bbe8-453b89deef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:55.246228 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878cf7-0a65-43bd-bbe8-453b89deef3c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:55.246733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878cf7-0a65-43bd-bbe8-453b89deef3c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:55.247216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878cf7-0a65-43bd-bbe8-453b89deef3c None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:55.248577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91878cf7-0a65-43bd-bbe8-453b89deef3c None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:55.250212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91878cf7-0a65-43bd-bbe8-453b89deef3c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:55.252537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 126/233] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:55 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:55.257977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:55.367514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5bdfa-9292-4b97-b96f-0362ea5bf6c3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:55.415228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917054d-7d15-4d15-b27c-980edd17c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:55.416367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917054d-7d15-4d15-b27c-980edd17c311 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:55.416639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917054d-7d15-4d15-b27c-980edd17c311 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:55.416915 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917054d-7d15-4d15-b27c-980edd17c311 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/e5d547f1548f40e18a9723578f675afd/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:55.420270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917054d-7d15-4d15-b27c-980edd17c311 None admin] RBAC: Authorizing `identity:create_grant(project_id=0718a24693104c6eada4aabe8e2ba6b6, user_id=e5d547f1548f40e18a9723578f675afd, role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:55.421717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917054d-7d15-4d15-b27c-980edd17c311 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:55.443169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:55.448032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 108/234] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:55 2022] PUT /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/e5d547f1548f40e18a9723578f675afd/roles/1d2b7e763c90438eab518a55d7589260 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:56.292467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc27cf-d0ff-4990-ba60-d49b2cad6984 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:56.292981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc27cf-d0ff-4990-ba60-d49b2cad6984 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:56.292981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc27cf-d0ff-4990-ba60-d49b2cad6984 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:56.294589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 127/235] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:56.299961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8be737-117b-4f92-9ea1-8312cede3474 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:56.300365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8be737-117b-4f92-9ea1-8312cede3474 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:56.300529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8be737-117b-4f92-9ea1-8312cede3474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:56.319151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6d8be737-117b-4f92-9ea1-8312cede3474 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:56.320361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6d8be737-117b-4f92-9ea1-8312cede3474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:56.412069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 109/236] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:56.702985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24a20-8c54-423f-9fd8-f3bcb93f96ee None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:56.703305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24a20-8c54-423f-9fd8-f3bcb93f96ee None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:56.703537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24a20-8c54-423f-9fd8-f3bcb93f96ee None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:56.705044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 128/237] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:56.717743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:56.718021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:56.718288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:56.739850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:56.741077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:56.804820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24a20-8c54-423f-9fd8-f3bcb93f96ee None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:56.819360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 110/238] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:56.935379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24a20-8c54-423f-9fd8-f3bcb93f96ee None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:56.951778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:56.972036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4624d5-101c-459b-ae73-cec7ffc72967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:56.972706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4624d5-101c-459b-ae73-cec7ffc72967 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:56.972985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4624d5-101c-459b-ae73-cec7ffc72967 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:56.973249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4624d5-101c-459b-ae73-cec7ffc72967 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:56.984797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4624d5-101c-459b-ae73-cec7ffc72967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ed4624d5-101c-459b-ae73-cec7ffc72967 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:56.989964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:56.990861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:56.991672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 129/239] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:30:56.997297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.089364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:57.140488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bfe4a-98ee-431a-bd92-15a71233154b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:57.162684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294de7f-8911-4a08-b71f-9b9d5659ee27 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.163014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294de7f-8911-4a08-b71f-9b9d5659ee27 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.163225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294de7f-8911-4a08-b71f-9b9d5659ee27 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.164926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 130/240] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:57.178733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.179016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.179337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.194195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:57.195003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.195414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.195842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.198228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:57.199916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:57.201330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:57.208746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:57.210162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 111/241] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:57.210643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:57.210765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.300659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 131/242] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:57.308962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.333043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549f087-0e85-4cf9-962d-caeceede8b02 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:57.359234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:57.375064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:57.375940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.376245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.376516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.377928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:57.379864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:57.381873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 112/243] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:30:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:57.389975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.504889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc9815-df84-4fdd-b0f1-ed9fc68f1728 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:57.550164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f05cb9-c4e8-4196-9bc7-ef404cfb1650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:57.550772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f05cb9-c4e8-4196-9bc7-ef404cfb1650 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.551046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f05cb9-c4e8-4196-9bc7-ef404cfb1650 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.551311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f05cb9-c4e8-4196-9bc7-ef404cfb1650 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.552416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f05cb9-c4e8-4196-9bc7-ef404cfb1650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:57.554049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f05cb9-c4e8-4196-9bc7-ef404cfb1650 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:57.561498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecbae8-4fa0-4d7c-984e-32bb59474888 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:57.573974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:57.581014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 113/244] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:30:57 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:30:57.638998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ec925-2f20-419c-8f64-510f9f41629d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.639347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ec925-2f20-419c-8f64-510f9f41629d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.639652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ec925-2f20-419c-8f64-510f9f41629d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.641058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 114/245] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:30:57.655834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.656137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.656430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.661676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:57.662331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.662612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.662887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] PATH_INFO: `/v3/users/20a522c20e224c44af35c3aea186bc2b` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.685783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:57.686985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:57.692584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] RBAC: Authorizing `identity:get_user(user_id=20a522c20e224c44af35c3aea186bc2b)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:57.694449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:57.697728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 132/246] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:57 2022] GET /identity/v3/users/20a522c20e224c44af35c3aea186bc2b => generated 297 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:57.698877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.742255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:57.777746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 115/247] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:30:57.777746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.812389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff970-05d2-4960-b2a2-5cf43daf2476 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:57.849267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:57.850244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.850721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.851196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:57.863094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:57.868850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:57.869753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:30:57.870568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:57.872891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 133/248] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:57.879454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:57.901512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48ca4c-7a06-46d6-8b90-35c4c89d6bdb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:57.914276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:57.945497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3248cae-9542-420f-9705-672c315853ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:57.946162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248cae-9542-420f-9705-672c315853ff None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:57.946438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248cae-9542-420f-9705-672c315853ff None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:57.946736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248cae-9542-420f-9705-672c315853ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.007214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.027676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b90177-1772-4089-bb8e-0d6234fb3d2c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.039092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3248cae-9542-420f-9705-672c315853ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.042409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3248cae-9542-420f-9705-672c315853ff None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.068579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.069906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.070582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.071329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.072813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:58.073307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.074883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.086194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 134/249] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:58.088176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.148787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 116/250] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:58.149647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3248cae-9542-420f-9705-672c315853ff None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.201242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.218892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491e8e0-0529-472a-ad0b-0183103aa6e3 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.247025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.258149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.259091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.259927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.260200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.273714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:58.281378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:58.282384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:30:58.283191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:30:58.284360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 135/251] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:58.287508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.298072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3248cae-9542-420f-9705-672c315853ff None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.417214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.417980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.418142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.418395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] PATH_INFO: `/v3/users/e5d547f1548f40e18a9723578f675afd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.420264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] RBAC: Authorizing `identity:get_user(user_id=e5d547f1548f40e18a9723578f675afd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.420504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c191a-3bab-44e4-82dd-89088b85a065 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.422518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.424797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 117/252] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/users/e5d547f1548f40e18a9723578f675afd => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:58.425775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.456731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ae96-2b49-4d15-ab7c-282e3aa8f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.457367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ae96-2b49-4d15-ab7c-282e3aa8f43c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.457640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ae96-2b49-4d15-ab7c-282e3aa8f43c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.457955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ae96-2b49-4d15-ab7c-282e3aa8f43c None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.459198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ae96-2b49-4d15-ab7c-282e3aa8f43c None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.460843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ae96-2b49-4d15-ab7c-282e3aa8f43c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.463112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 136/253] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:58.464711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c49f51-4432-40f8-8ea4-029892269fc8 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.464990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c49f51-4432-40f8-8ea4-029892269fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.465256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c49f51-4432-40f8-8ea4-029892269fc8 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.466453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 137/254] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:30:58.471190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c49f51-4432-40f8-8ea4-029892269fc8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.488638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.541206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e979c66-2411-4261-b42d-48328701dfad None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.578608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.579670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.579670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.579864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.581391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:58.581391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.582099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.583991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c49f51-4432-40f8-8ea4-029892269fc8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.592349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 118/255] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:58.593251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.603687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.636578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.637568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.638352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.639733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.716281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.717181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1304f-319a-480e-980d-d31c58e335f1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.718478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.734112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.753804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.754411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.754682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.754964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.755703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] RBAC: Adding query filter params (role.id=9a827b3ce4b243f494208e3f97029a8d, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=20a522c20e224c44af35c3aea186bc2b) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:58.756006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.757435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e53e9-5e26-4254-bbab-56d606904f86 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.764000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 119/256] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/role_assignments?user.id=20a522c20e224c44af35c3aea186bc2b&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=9a827b3ce4b243f494208e3f97029a8d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:58.765033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.765318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.765585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.785562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:30:58.786504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:30:58.835904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 138/257] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:30:58.837284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.907108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.916846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 120/258] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:30:58.917726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:58.951116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3496d-1088-4e74-883a-eedffe7590ab None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:58.967862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:58.991464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:58.992517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:58.992997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:58.993468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] PATH_INFO: `/v3/users/d78b599ca2294fbb81c65eb58931ca02` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:58.994904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] RBAC: Authorizing `identity:get_user(user_id=d78b599ca2294fbb81c65eb58931ca02)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:58.996922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:58.998824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 139/259] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/users/d78b599ca2294fbb81c65eb58931ca02 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:59.005685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:59.126339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e0ed9-92f2-485b-99b0-7555a5778acf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.170774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.171811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.172289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.172752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.174071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:59.175772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:59.177797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 121/260] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:58 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:59.188590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:59.212550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62d122-c871-4851-b3e1-1802baf09ada None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.273323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:59.276724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.277364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.277676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.277962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.282535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:59.284024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:59.285971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 140/261] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:30:59 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:30:59.292799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:59.321565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfea5a-c61f-424d-a062-d2e371bb4122 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.380819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.381364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.381640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.381884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:59.381983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.382639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=e5d547f1548f40e18a9723578f675afd) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:59.382949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:59.385330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:59.398061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 122/262] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:30:59 2022] GET /identity/v3/role_assignments?user.id=e5d547f1548f40e18a9723578f675afd&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:59.398744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:59.449559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61099197-0b2e-40b2-8bde-ecc1a90418ed None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.511468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e49f6-e0fe-4f44-b89d-e71147321ffd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.516412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.517419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.517883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.518338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/d78b599ca2294fbb81c65eb58931ca02/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.522736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0718a24693104c6eada4aabe8e2ba6b6, user_id=d78b599ca2294fbb81c65eb58931ca02, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:59.524466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:59.544136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:59.554839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 141/263] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:30:59 2022] PUT /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/d78b599ca2294fbb81c65eb58931ca02/roles/87f1831257824560813f954348534540 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:30:59.555294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.556054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.556840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.557117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.581549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:30:59.587065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:30:59.588671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:30:59.589456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 123/264] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:30:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:30:59.595726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:59.752450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85400298-6d93-4b2f-8489-2c0cdeaefbe1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.774027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:30:59.810255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.810869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.811207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.811501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.812194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:30:59.812540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:59.813933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:59.825329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 142/265] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:30:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:59.832666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:30:59.951822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cd91d-ff32-4576-bed3-420393aca494 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:30:59.987124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:30:59.987796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:30:59.988133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:30:59.988406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] PATH_INFO: `/v3/users/e5d547f1548f40e18a9723578f675afd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:30:59.989560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] RBAC: Authorizing `identity:get_user(user_id=e5d547f1548f40e18a9723578f675afd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:30:59.991048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:30:59.992684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 124/266] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:30:59 2022] GET /identity/v3/users/e5d547f1548f40e18a9723578f675afd => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:30:59.999036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:00.127990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981c102-215a-42d1-a794-4ba6751295fd None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:00.173333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:00.173974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:00.174250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:00.174522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:00.217306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:00.250362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:00.251850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:00.381864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 143/267] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:00.387863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:00.452904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:00.496454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4eb136-94ee-4cf1-9fe2-eb1dc130a114 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:00.531025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a45a83-1dc4-4a30-925f-9e9d7f49a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:00.531693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45a83-1dc4-4a30-925f-9e9d7f49a96b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:00.532011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45a83-1dc4-4a30-925f-9e9d7f49a96b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:00.532290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45a83-1dc4-4a30-925f-9e9d7f49a96b None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:00.533511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a45a83-1dc4-4a30-925f-9e9d7f49a96b None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:00.534986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a45a83-1dc4-4a30-925f-9e9d7f49a96b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:00.536986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 125/268] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:00 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:00.543640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:00.651092 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:00.689107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229721d0-1e5f-4483-8bb8-79c89b473db0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:00.744731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:00.745683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:00.746150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:00.746660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:00.747611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] RBAC: Adding query filter params (role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=e5d547f1548f40e18a9723578f675afd) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:00.748138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:00.749755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ed97a-6b74-4594-a7e1-7b2e388c5ed8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:00.756486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 144/269] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:00 2022] GET /identity/v3/role_assignments?user.id=e5d547f1548f40e18a9723578f675afd&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=1d2b7e763c90438eab518a55d7589260 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:00.921913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597c63e-67da-451d-9155-bdbce1534a02 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:00.922287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597c63e-67da-451d-9155-bdbce1534a02 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:00.922486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597c63e-67da-451d-9155-bdbce1534a02 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:00.924207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 126/270] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:00.929743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c1ca2-18a5-4323-b7db-a433cf1f7de6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:00.930337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c1ca2-18a5-4323-b7db-a433cf1f7de6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:00.930830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c1ca2-18a5-4323-b7db-a433cf1f7de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:00.960746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8a3c1ca2-18a5-4323-b7db-a433cf1f7de6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:00.962908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8a3c1ca2-18a5-4323-b7db-a433cf1f7de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:01.050579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 145/271] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:01.075380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5ea31-3c1d-4c33-9bd1-46b8457664cb None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.075674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5ea31-3c1d-4c33-9bd1-46b8457664cb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.075948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5ea31-3c1d-4c33-9bd1-46b8457664cb None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.077286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 127/272] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:01.082636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.082924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.083195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.105143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:01.107112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:01.171694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:01.182306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 146/273] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:01.368063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5ea31-3c1d-4c33-9bd1-46b8457664cb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:01.482055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:01.485618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5ea31-3c1d-4c33-9bd1-46b8457664cb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:01.515991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:01.525978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:01.526856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.527172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.527550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.528780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:01.530969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:01.533124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 128/274] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:01.540770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:01.595415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a755-3290-46ac-b8d2-7366aece0538 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:01.635611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:01.636508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.636961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.637411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.655082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:01.660010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:01.660885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:01.667077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 147/275] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:01.669654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:01.688559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c723e-32e3-4c2d-ac62-894019727aea None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:01.733573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a8350-ba1e-473d-bd2b-81b55ec01023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:01.734479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8350-ba1e-473d-bd2b-81b55ec01023 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.734818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8350-ba1e-473d-bd2b-81b55ec01023 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.735093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8350-ba1e-473d-bd2b-81b55ec01023 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.735772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a8350-ba1e-473d-bd2b-81b55ec01023 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:01.737219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a8350-ba1e-473d-bd2b-81b55ec01023 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:01.740139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:01.761482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 129/276] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:01 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:31:01.764812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3271c9-abd9-4061-b838-d8b429e9fd11 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.767526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3271c9-abd9-4061-b838-d8b429e9fd11 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.767782 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3271c9-abd9-4061-b838-d8b429e9fd11 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.771574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 130/277] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:01.775821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.776514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.776949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.800743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:01.801685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1b0d0-f586-4bba-9fa5-67ec8d2110e4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:01.802531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:01.842963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:01.843786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:01.844146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:01.844517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:01.845339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:01.845864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:01.851542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:01.857304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 148/278] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:01.862974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:01.865808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:01.903821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 131/279] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:01.985876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae8808-db57-4bfd-94c5-7089af865723 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:02.005656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:02.030687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:02.031317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.031619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.031892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] PATH_INFO: `/v3/users/cf7fceb719ef49b08d9aaeb5cb353064` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.047786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7fceb719ef49b08d9aaeb5cb353064)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:02.049235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:02.051104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 149/280] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:01 2022] GET /identity/v3/users/cf7fceb719ef49b08d9aaeb5cb353064 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:02.057209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:02.172900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046800-82cb-4263-bbfb-1e9b1274af63 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:02.191804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:02.216546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:02.217670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.217670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.217670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.218443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, user.id=cf7fceb719ef49b08d9aaeb5cb353064, scope.system=all) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:02.218750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:02.220639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59969a-1826-46d5-b033-e11e00ae6149 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:02.226351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 132/281] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 00:31:02 2022] GET /identity/v3/role_assignments?user.id=cf7fceb719ef49b08d9aaeb5cb353064&scope.system=all&role.id=87f1831257824560813f954348534540 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:02.319116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ca445-f787-4c1b-bbdf-aeab193d3864 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.319512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ca445-f787-4c1b-bbdf-aeab193d3864 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.319684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ca445-f787-4c1b-bbdf-aeab193d3864 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.321062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 133/282] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:02.322673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e906a-18f2-4f02-8e67-44108e76d266 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:02.339137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.339834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.340307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.365175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:02.365175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.365609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.365783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.372497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:02.373946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:02.393828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:02.401044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:02.401961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:02.402757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:02.403859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 150/283] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:02.409108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:02.438580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:02.451413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:02.479091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 134/284] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:02.533069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aa4d8-2274-4d4e-b3c4-02051f893bf4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:02.576757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:02.577382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.577658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.577935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.578604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:02.578903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:02.580304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:02.589750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 151/285] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:02.595349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:02.725887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97de4-63e1-4cb5-955b-267f419692e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:02.761145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:02.761776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.762050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.762320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] PATH_INFO: `/v3/users/20a522c20e224c44af35c3aea186bc2b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.763446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] RBAC: Authorizing `identity:get_user(user_id=20a522c20e224c44af35c3aea186bc2b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:02.764986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:02.766570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 135/286] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:02 2022] GET /identity/v3/users/20a522c20e224c44af35c3aea186bc2b => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:02.772186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:02.877393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8942-9b8a-4bce-8744-fb055f0f9eb8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:02.879083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:02.913054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:02.913687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:02.913964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:02.914234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:02.915445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:02.916898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:02.918830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 152/287] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:02 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:02.924467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:02.972899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:03.005140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:03.063507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4db9c-bcc7-4808-beef-3511c6f76b5f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:03.130832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:03.131527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:03.131815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:03.132248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/20a522c20e224c44af35c3aea186bc2b/roles/9a827b3ce4b243f494208e3f97029a8d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:03.137513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] RBAC: Authorizing `identity:create_grant(project_id=0718a24693104c6eada4aabe8e2ba6b6, user_id=20a522c20e224c44af35c3aea186bc2b, role_id=9a827b3ce4b243f494208e3f97029a8d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:03.141339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:03.168914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 136/288] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:31:02 2022] PUT /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/users/20a522c20e224c44af35c3aea186bc2b/roles/9a827b3ce4b243f494208e3f97029a8d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:03.205784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66565-5d82-44cf-a0ff-d80e706c24fc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:03.253743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:03.254396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:03.254676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:03.254950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:03.272312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:03.278074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:03.279008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:03.279841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 153/289] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:03.285953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:03.413973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc990-8867-4f60-aa66-27afa32d076a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:03.417046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:03.451868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:03.452499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:03.452777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:03.453110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:03.453813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:03.454176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:03.455707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:03.465722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 137/290] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:03.471284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:03.520188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:03.613032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe497a6-7a8b-4237-a433-113b1ef65745 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:03.669566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:03.671153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:03.671782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:03.672274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] PATH_INFO: `/v3/users/d78b599ca2294fbb81c65eb58931ca02` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:03.676628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] RBAC: Authorizing `identity:get_user(user_id=d78b599ca2294fbb81c65eb58931ca02)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:03.677586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:03.679189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 154/291] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:03 2022] GET /identity/v3/users/d78b599ca2294fbb81c65eb58931ca02 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:03.685599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:03.806642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae9903-2164-4703-8516-99317151df91 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:03.847508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:03.848116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:03.848392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:03.848664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:03.891502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:03.927071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:03.928581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:04.033161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 138/292] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:04.039794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:04.158794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:04.166088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6022f2-e0d6-4f36-abbf-69659d030f34 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:04.208214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967d4e6-7ab9-423b-8f8b-dc19d6c3fd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:04.208847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967d4e6-7ab9-423b-8f8b-dc19d6c3fd1f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:04.209189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967d4e6-7ab9-423b-8f8b-dc19d6c3fd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:04.209457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967d4e6-7ab9-423b-8f8b-dc19d6c3fd1f None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:04.210630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967d4e6-7ab9-423b-8f8b-dc19d6c3fd1f None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:04.212202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967d4e6-7ab9-423b-8f8b-dc19d6c3fd1f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:04.214008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 155/293] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:04 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:04.219893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:04.285983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:04.334225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885aa52-de17-4bbc-b04a-4b17378bbb2c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:04.377779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:04.378406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:04.378681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:04.378947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:04.379745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=d78b599ca2294fbb81c65eb58931ca02) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:04.380057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:04.381666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dc510-b5cf-45da-8936-d5a73873da1e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:04.388849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 139/294] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:04 2022] GET /identity/v3/role_assignments?user.id=d78b599ca2294fbb81c65eb58931ca02&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=87f1831257824560813f954348534540 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:04.525751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bdecf-c42e-4b90-b9a9-ea7acfd3cbd0 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:04.526096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bdecf-c42e-4b90-b9a9-ea7acfd3cbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:04.526266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bdecf-c42e-4b90-b9a9-ea7acfd3cbd0 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:04.528728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 156/295] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:04.536237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a09172-7229-4f74-b17f-47bd5ed9001e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:04.536521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a09172-7229-4f74-b17f-47bd5ed9001e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:04.536796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a09172-7229-4f74-b17f-47bd5ed9001e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:04.573762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f3a09172-7229-4f74-b17f-47bd5ed9001e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:04.575040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f3a09172-7229-4f74-b17f-47bd5ed9001e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:04.663932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 140/296] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:04.968283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69422bb-4bbe-4492-bfb8-6f2358305d52 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:04.968539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69422bb-4bbe-4492-bfb8-6f2358305d52 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:04.968809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69422bb-4bbe-4492-bfb8-6f2358305d52 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:04.970417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 157/297] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:04.976097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:04.976565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:04.976639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.003260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:05.004486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:05.022336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6496e22-2b12-4a6e-9246-4641ab80b9f1 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.022641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6496e22-2b12-4a6e-9246-4641ab80b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.022882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6496e22-2b12-4a6e-9246-4641ab80b9f1 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.024284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 158/298] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:05.034668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.035525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.036266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.058780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:05.061242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:05.079472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:05.098062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 141/299] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:05.098901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.149955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 159/300] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:05.214378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766581-6878-4920-b68b-f6799a2c1f9c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.254123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.255106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.255629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.256109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.257393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:05.259965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:05.261768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 142/301] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:05.269355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.360846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:05.369755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3ae81-fc2a-456d-8615-59ddcf9ff314 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.386588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.405374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.405985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.406261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.406533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] PATH_INFO: `/v3/groups` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.407168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:05.408688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:05.429379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 160/302] 10.222.0.15 () {60 vars in 1134 bytes} [Thu Sep 15 00:31:05 2022] POST /identity/v3/groups => generated 229 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:31:05.439891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.468172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:05.528495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7421e4-5be8-425e-9a18-1a4c36923c78 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.574681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92adbc7-de35-4698-9012-b0a7098cd0f8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.582114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.583345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.583847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.584306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.600095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:05.608508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:05.609480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:05.610304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:05.617454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 143/303] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:05.619524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.624724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.625554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.626009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.626277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.637866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:05.645717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:05.646677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:05.647616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 161/304] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:05.654244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.672416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:05.745056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4282ead-e6e3-4c57-93a8-2639436245f2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.764035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:05.782947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eb223-cf6f-412f-8479-cd229c41d9fb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.783727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.784655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.785222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.785688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.786580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:05.787110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:05.788759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:05.798906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 144/305] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:05.804782 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.825180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.826123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.826594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.827067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.828059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:05.828575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:05.830233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:05.841037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 162/306] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:05.851565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.915922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdb27d-083d-4f79-986d-d26f05b24532 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:05.952507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:05.953130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:05.953402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:05.953673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] PATH_INFO: `/v3/users/cf7fceb719ef49b08d9aaeb5cb353064` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:05.954049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:05.954822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7fceb719ef49b08d9aaeb5cb353064)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:05.956283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:05.963085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 145/307] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/users/cf7fceb719ef49b08d9aaeb5cb353064 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:05.964737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:05.979758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5cc08-ee52-4552-9b45-c49f142b17f6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:06.015279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:06.017907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af78ab3-5339-40fb-b34d-daa79a0305f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:06.018586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78ab3-5339-40fb-b34d-daa79a0305f7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.018858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78ab3-5339-40fb-b34d-daa79a0305f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.019698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78ab3-5339-40fb-b34d-daa79a0305f7 None admin] PATH_INFO: `/v3/users/1266ef9190f642b7870876d39581a05c` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.037226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af78ab3-5339-40fb-b34d-daa79a0305f7 None admin] RBAC: Authorizing `identity:get_user(user_id=1266ef9190f642b7870876d39581a05c)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:06.038626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af78ab3-5339-40fb-b34d-daa79a0305f7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:06.041116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 163/308] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:05 2022] GET /identity/v3/users/1266ef9190f642b7870876d39581a05c => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:06.045915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295928-a799-4e08-82dd-8d0bde895819 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.045915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295928-a799-4e08-82dd-8d0bde895819 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.045915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295928-a799-4e08-82dd-8d0bde895819 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.045915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 164/309] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:06.046767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd295928-a799-4e08-82dd-8d0bde895819 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:06.074943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344086a5-63cc-48a4-b653-9c96fd8f4623 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:06.111465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba236f0-df54-4d11-bedb-4a4182d26796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:06.112163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba236f0-df54-4d11-bedb-4a4182d26796 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.112431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba236f0-df54-4d11-bedb-4a4182d26796 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.112711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba236f0-df54-4d11-bedb-4a4182d26796 None admin] PATH_INFO: `/v3/system/users/cf7fceb719ef49b08d9aaeb5cb353064/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.114116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba236f0-df54-4d11-bedb-4a4182d26796 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cf7fceb719ef49b08d9aaeb5cb353064, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:06.115564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba236f0-df54-4d11-bedb-4a4182d26796 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:06.125032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 146/310] 10.222.0.15 () {58 vars in 1334 bytes} [Thu Sep 15 00:31:05 2022] PUT /identity/v3/system/users/cf7fceb719ef49b08d9aaeb5cb353064/roles/87f1831257824560813f954348534540 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:06.132481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.132481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.132481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.154973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:06.156242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:06.171244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd295928-a799-4e08-82dd-8d0bde895819 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:06.203799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:06.209727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:06.210683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.211113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.211584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.213012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] RBAC: Adding query filter params (role.id=9a827b3ce4b243f494208e3f97029a8d, user.id=1266ef9190f642b7870876d39581a05c, scope.system=all) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:06.213426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:06.215054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:06.221408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 165/311] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 00:31:06 2022] GET /identity/v3/role_assignments?user.id=1266ef9190f642b7870876d39581a05c&scope.system=all&role.id=9a827b3ce4b243f494208e3f97029a8d => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:06.254222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 147/312] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:06.531266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:06.637914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3afa62-7ff3-4891-ab79-caa488805dc0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:06.674007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:06.674637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.674913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.675176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.685907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:06.690816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:06.691777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:06.692671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 166/313] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:06.698045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:06.743125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:06.805361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af289f91-4dfd-4e85-8798-764a58945bd9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:06.841092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:06.841702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:06.841976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:06.842289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:06.842962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:06.843260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:06.844676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:06.853002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 148/314] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:06.858665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:06.917986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:06.964247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3edc-0851-4b1f-9655-6472980d37a0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:06.999907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:07.000528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:07.000803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:07.001068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] PATH_INFO: `/v3/users/20a522c20e224c44af35c3aea186bc2b` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:07.002173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] RBAC: Authorizing `identity:get_user(user_id=20a522c20e224c44af35c3aea186bc2b)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:07.003609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:07.005176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 167/315] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:06 2022] GET /identity/v3/users/20a522c20e224c44af35c3aea186bc2b => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:07.010971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:07.124717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1325ac-4026-4cca-8fba-a7629bdddd67 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:07.127091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:07.164390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:07.165013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:07.165326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:07.165598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:07.234528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:07.236044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:07.337949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 149/316] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:07.343736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:07.449246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d602a7-21eb-4f37-b411-77410014411b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:07.473534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:07.484755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431f057-3997-4ddb-a411-d4b0d3c0a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:07.485740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431f057-3997-4ddb-a411-d4b0d3c0a855 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:07.486185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431f057-3997-4ddb-a411-d4b0d3c0a855 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:07.486621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431f057-3997-4ddb-a411-d4b0d3c0a855 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:07.487979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431f057-3997-4ddb-a411-d4b0d3c0a855 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:07.489701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431f057-3997-4ddb-a411-d4b0d3c0a855 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:07.495789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 168/317] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:07 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:07.498509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:07.520711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:07.600361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c408c-8a72-4b83-816b-ae1cf7130f24 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:07.634780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:07.635592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:07.636002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:07.636311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:07.637056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] RBAC: Adding query filter params (role.id=9a827b3ce4b243f494208e3f97029a8d, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6, user.id=20a522c20e224c44af35c3aea186bc2b) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:07.637362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:07.638848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3fc22-2ce3-450b-906f-e74753360f5e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:07.645664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 150/318] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:07 2022] GET /identity/v3/role_assignments?user.id=20a522c20e224c44af35c3aea186bc2b&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=9a827b3ce4b243f494208e3f97029a8d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:07.770346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f8e69-f1f2-4eb4-b976-890ab97854db None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:07.770524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f8e69-f1f2-4eb4-b976-890ab97854db None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:07.770805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f8e69-f1f2-4eb4-b976-890ab97854db None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:07.772663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 169/319] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:07.779255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:07.779592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:07.779827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:07.802481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:07.804098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:07.872106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:07.878036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 151/320] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:08.151833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f8e69-f1f2-4eb4-b976-890ab97854db None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:08.253638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f8e69-f1f2-4eb4-b976-890ab97854db None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:08.290868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57778f9-a761-4135-ba37-60947fb90a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:08.291534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57778f9-a761-4135-ba37-60947fb90a6c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.291811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57778f9-a761-4135-ba37-60947fb90a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.292084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57778f9-a761-4135-ba37-60947fb90a6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.293244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57778f9-a761-4135-ba37-60947fb90a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:08.294785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57778f9-a761-4135-ba37-60947fb90a6c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:08.296838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 170/321] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:08.302240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:08.396274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0f4b2-f129-4f42-acd4-4769ae2de251 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.396796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0f4b2-f129-4f42-acd4-4769ae2de251 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.396796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0f4b2-f129-4f42-acd4-4769ae2de251 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.398337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 171/322] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:08.403178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.403407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.403710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.408584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf1e2e-9ce5-4aaa-b67b-0c297a9baa78 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:08.421622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:08.422803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:08.444993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da4666-ca89-4f79-b0ef-0e327896850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:08.445609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4666-ca89-4f79-b0ef-0e327896850f None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.445887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4666-ca89-4f79-b0ef-0e327896850f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.446159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4666-ca89-4f79-b0ef-0e327896850f None admin] PATH_INFO: `/v3/groups` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.447383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da4666-ca89-4f79-b0ef-0e327896850f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:08.448888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da4666-ca89-4f79-b0ef-0e327896850f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:08.466430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 152/323] 10.222.0.15 () {60 vars in 1134 bytes} [Thu Sep 15 00:31:08 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:08.483329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:08.484202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3efdd2-5628-4b18-8ce3-7135ba15518f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.484503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3efdd2-5628-4b18-8ce3-7135ba15518f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.484826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3efdd2-5628-4b18-8ce3-7135ba15518f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.486517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 153/324] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:08.492406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.492688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.492967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.511528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 172/325] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:08.513407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:08.514625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:08.575028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:08.592963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 154/326] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:08.768631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:08.861047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:08.879011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cd128-1c65-4cca-a71b-cc4cee1bb6c7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:08.928996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:08.929819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:08.930098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:08.930369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:08.946071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:08.951035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:08.951949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:08.952732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 173/327] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:08.957957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:08.983996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ff84-e9f8-4f48-a369-35b7a3f3966e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.027765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:09.032683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.033667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.033952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.034227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.048977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:09.055253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:09.056603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:09.057921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 155/328] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:09.064620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:09.092109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a4645-bc1e-4c5e-85a5-9fe844bb83a6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.134215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:09.136501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.137445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.138073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.138847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.140111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:09.140734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:09.142919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:09.155395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 174/329] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:09.163141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:09.192074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f2a67-0be2-4bbb-a465-557f7c630606 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.241449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.242385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.242941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.243428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.244346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:09.244853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:09.246632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:09.259498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 156/330] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:09.267382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:09.298738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323321d8-22e5-4dda-9063-171bacd4fa7c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.318621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:09.352339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.353744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.354027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.354300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] PATH_INFO: `/v3/users/cf7fceb719ef49b08d9aaeb5cb353064` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.356122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7fceb719ef49b08d9aaeb5cb353064)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:09.357911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:09.359363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 175/331] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:09 2022] GET /identity/v3/users/cf7fceb719ef49b08d9aaeb5cb353064 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:09.365689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:09.409803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55f1bb-fc31-4bbd-91a2-fdb0becc9821 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.422553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:09.456526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.457089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.457366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.457642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] PATH_INFO: `/v3/users/1266ef9190f642b7870876d39581a05c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.458787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] RBAC: Authorizing `identity:get_user(user_id=1266ef9190f642b7870876d39581a05c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:09.460751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:09.461857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 157/332] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:09 2022] GET /identity/v3/users/1266ef9190f642b7870876d39581a05c => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:09.467811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:09.494176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9817e-3bff-4ee5-a73f-c18208b8f6a5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.531006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.532108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.532614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.533073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.533995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] RBAC: Adding query filter params (role.id=87f1831257824560813f954348534540, user.id=cf7fceb719ef49b08d9aaeb5cb353064, scope.system=all) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:09.534493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:09.536129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc936-b8c9-4903-b0c4-c66fed6ab82e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:09.550600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 176/333] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 00:31:09 2022] GET /identity/v3/role_assignments?user.id=cf7fceb719ef49b08d9aaeb5cb353064&scope.system=all&role.id=87f1831257824560813f954348534540 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:09.604539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e0b2a-4e5b-4739-a966-33ffd7060d89 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:09.650731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a109b-1305-4c5b-8d88-9099de2b277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:09.651385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a109b-1305-4c5b-8d88-9099de2b277f None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:09.655819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a109b-1305-4c5b-8d88-9099de2b277f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:09.656123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a109b-1305-4c5b-8d88-9099de2b277f None admin] PATH_INFO: `/v3/system/users/1266ef9190f642b7870876d39581a05c/roles/9a827b3ce4b243f494208e3f97029a8d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:09.657711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a109b-1305-4c5b-8d88-9099de2b277f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1266ef9190f642b7870876d39581a05c, role_id=9a827b3ce4b243f494208e3f97029a8d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:09.659116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a109b-1305-4c5b-8d88-9099de2b277f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:09.676568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 158/334] 10.222.0.15 () {58 vars in 1334 bytes} [Thu Sep 15 00:31:09 2022] PUT /identity/v3/system/users/1266ef9190f642b7870876d39581a05c/roles/9a827b3ce4b243f494208e3f97029a8d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:31:10.991882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9937f7-e912-41d6-845a-e68a2f8be2cb None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:10.992942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9937f7-e912-41d6-845a-e68a2f8be2cb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:10.993479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9937f7-e912-41d6-845a-e68a2f8be2cb None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:10.995252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 177/335] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:11.003073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:11.003989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:11.004476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:11.025281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:11.026857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:11.067141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:11.106122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 159/336] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:11.412541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9937f7-e912-41d6-845a-e68a2f8be2cb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:11.461079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:11.533560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9937f7-e912-41d6-845a-e68a2f8be2cb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:11.578270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c8756-0e32-49ef-afe5-14a422b7fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:11.578907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c8756-0e32-49ef-afe5-14a422b7fc02 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:11.579186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c8756-0e32-49ef-afe5-14a422b7fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:11.579463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c8756-0e32-49ef-afe5-14a422b7fc02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:11.594118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c8756-0e32-49ef-afe5-14a422b7fc02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b51c8756-0e32-49ef-afe5-14a422b7fc02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:11.599131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:11.600116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:11.600922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 178/337] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:11.607191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:11.638747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba6d2-dc76-46f7-a908-e4be4b5c8adc None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:11.639082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba6d2-dc76-46f7-a908-e4be4b5c8adc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:11.639293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba6d2-dc76-46f7-a908-e4be4b5c8adc None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:11.642412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 179/338] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:11.649852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:11.650164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:11.650404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:11.692514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:11.693737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:11.788922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218d68f-6f9f-412c-9d7e-031501a195ad None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:11.826165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 180/339] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:11.857037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:11.857736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:11.858021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:11.858294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:11.858982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:11.859286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:11.860709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fed33-d004-4e85-9ab3-19711a229278 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:11.875114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 160/340] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:11.880718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:11.926408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:12.016922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f76b43-9e52-4f67-a062-b3b06a441dd1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:12.021496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045acd7-1e4b-4b64-84e9-6992bc6d6e2c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.021964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045acd7-1e4b-4b64-84e9-6992bc6d6e2c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.022038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045acd7-1e4b-4b64-84e9-6992bc6d6e2c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.023619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 161/341] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:12.043169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.043526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.043757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.072679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:12.073877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:12.081654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:12.082275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.082556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.082826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.136723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:12.213351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:12.214808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:12.224027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 162/342] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:12.261817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:12.332560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:12.365140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 181/343] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:12.371302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:12.536987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934662f-7abe-44b8-80a0-8c574c0009f1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:12.572621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a74ac-ff77-4515-82c8-15969875aac3 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:12.593579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:12.594143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.594417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.594684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.608500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:12.616113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:12.616996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:12.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 163/344] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:12.624173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:12.660286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:12.660927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:12.680206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:12.681386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.681665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.681936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.683136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:12.684625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:12.686540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 182/345] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:12.687321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:12.747973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d5d30-4e26-45a6-a1f3-867c32184768 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:12.785155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:12.785768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.786044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.786313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.786986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:12.787285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:12.788717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:12.797421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 164/346] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:12.798081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:12.842371 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8934-5725-4e4a-9f51-60a29a1c7aa7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:12.889924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:12.890494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.890767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.891036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.896112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:12.908584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:12.913759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55ae5-7fef-4f97-b429-477a78e1f011 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:12.917892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:12.918767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:12.919636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:12.924969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 183/347] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:12.925801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:12.951233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:12.951902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:12.952181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:12.952456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:12.958689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:12.960183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:12.961818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 165/348] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:12.962605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:13.011901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:13.061455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a7101-fd99-434c-bb9d-09d0fa50bf16 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.069397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167c048-f799-447a-a63d-d0ba2e1b97cf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.105306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:13.105916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.106194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.106468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.107138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:13.107443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:13.108833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:13.118488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 166/349] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:13.119119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:13.121048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b928a-a3f0-4f73-b3d8-04d054101017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:13.121681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b928a-a3f0-4f73-b3d8-04d054101017 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.121962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b928a-a3f0-4f73-b3d8-04d054101017 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.122234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b928a-a3f0-4f73-b3d8-04d054101017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.136683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:13.232155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b928a-a3f0-4f73-b3d8-04d054101017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:13.233567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b928a-a3f0-4f73-b3d8-04d054101017 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:13.237205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6059cf7-6b74-405e-8d68-0fcd77618643 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.274853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:13.275459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.275779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.276050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.276786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=87f1831257824560813f954348534540, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:13.277092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:13.278544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:13.284746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 167/350] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:13 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=87f1831257824560813f954348534540 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:13.285577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:13.371358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:13.372851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:13.398528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8fe4-a78d-4941-b052-69acd3be9e10 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.405618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 184/351] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:13.406921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c8bf-69fd-4dbf-8fbc-b7ab4d6794b1 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.407206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c8bf-69fd-4dbf-8fbc-b7ab4d6794b1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.407498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c8bf-69fd-4dbf-8fbc-b7ab4d6794b1 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.408796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 185/352] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:13.414071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49c8bf-69fd-4dbf-8fbc-b7ab4d6794b1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:13.474741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:13.475364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.475675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.475947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.592328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:13.593987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:13.599360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49c8bf-69fd-4dbf-8fbc-b7ab4d6794b1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.662238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e71695-f05b-4f5f-94b3-3b3f8790820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:13.664738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71695-f05b-4f5f-94b3-3b3f8790820b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.669434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71695-f05b-4f5f-94b3-3b3f8790820b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.669969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71695-f05b-4f5f-94b3-3b3f8790820b None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.672329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e71695-f05b-4f5f-94b3-3b3f8790820b None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:13.679521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e71695-f05b-4f5f-94b3-3b3f8790820b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:13.679521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 186/353] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:13 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:13.679890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.680103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.680373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.723022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:13.724235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:13.752576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:13.755090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:13.786489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 168/354] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:13.787390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:13.838601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 187/355] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:13.839471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:13.941753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a130cd-1798-4ff3-9357-4a7131bec115 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.985611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f8338-5a4c-404e-a5d6-cedafda2d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:13.986588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8338-5a4c-404e-a5d6-cedafda2d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.987083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8338-5a4c-404e-a5d6-cedafda2d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.987582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8338-5a4c-404e-a5d6-cedafda2d5d1 None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:13.988894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f8338-5a4c-404e-a5d6-cedafda2d5d1 None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:13.990559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f8338-5a4c-404e-a5d6-cedafda2d5d1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:13.992663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 169/356] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:13 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:13.993969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92615-f30e-4baf-b42e-055d15f91da9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:13.994778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c18ce-d37c-432e-80db-8669454205a6 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:13.995285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c18ce-d37c-432e-80db-8669454205a6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:13.999882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c18ce-d37c-432e-80db-8669454205a6 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.001730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 170/357] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:14.003115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3d6de-57d3-4afb-a554-f0a09190aa4c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.003628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3d6de-57d3-4afb-a554-f0a09190aa4c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.004105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3d6de-57d3-4afb-a554-f0a09190aa4c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.005729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 171/358] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:14.012414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3d6de-57d3-4afb-a554-f0a09190aa4c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.039505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae54250-a7e3-45a4-8584-90ddf87242d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.040694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae54250-a7e3-45a4-8584-90ddf87242d1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.040972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae54250-a7e3-45a4-8584-90ddf87242d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.041241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae54250-a7e3-45a4-8584-90ddf87242d1 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.042475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae54250-a7e3-45a4-8584-90ddf87242d1 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:14.044139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae54250-a7e3-45a4-8584-90ddf87242d1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:14.050562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 188/359] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:13 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:14.051974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439f20e-ef4a-41c1-ad01-e8c10754730e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.052283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439f20e-ef4a-41c1-ad01-e8c10754730e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.053279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439f20e-ef4a-41c1-ad01-e8c10754730e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.066554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:14.077750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c439f20e-ef4a-41c1-ad01-e8c10754730e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:14.079572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c439f20e-ef4a-41c1-ad01-e8c10754730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:14.099637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:14.143402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3d6de-57d3-4afb-a554-f0a09190aa4c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:14.165381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 189/360] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:14.166948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.167234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.167553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.187128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:14.188365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:14.192680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.193582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.194116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.194580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.195584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:14.196110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:14.198462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:14.206019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 172/361] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=1d2b7e763c90438eab518a55d7589260 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:14.207318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.271263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 190/362] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:14.272087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.395886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c66a-9ed6-43cc-98b8-befee92b75fb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:14.411874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:14.448019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.448590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.448869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.449144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.450302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:14.451773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:14.453031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:14.454608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 173/363] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:14.460700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.491095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765a5c3-2007-42e9-83d3-7fbb17cceabd None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:14.539068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.539744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.540555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.540825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.554897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:14.560669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:14.561622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:14.562429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:14.563019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 191/364] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:14.564436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.618743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cf2cd-13d6-44e5-98e5-8c268b8ca7f4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:14.662099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.663379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.663887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.664311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.665078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=87f1831257824560813f954348534540, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:14.665385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:14.666825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:14.673522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 174/365] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=87f1831257824560813f954348534540 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:14.674229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.719455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8905a-80e0-4151-bf0b-f032a51b7a8a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:14.728315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:14.758148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.760011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.760509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.760975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.773080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:14.779246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:14.780257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:14.781123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:14.786372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 192/366] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:14.786372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.788333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:14.829470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34180316-a4d8-4f82-9e40-cf95ae396e25 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:14.900599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:14.901235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:14.901662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:14.901936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:14.903753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:14.904064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:14.905513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:14.927715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 175/367] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:14.929501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:14.966258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc678-96b4-4464-90ca-d544e6f2a750 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:15.041352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:15.041911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:15.042186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:15.042451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:15.058064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:15.071514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:15.072565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 00:31:15.073362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:15.075770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 193/368] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:15.076812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:15.100909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e58c-d0f6-41ee-ac45-e4f33ccd6f5b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:15.103145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:15.141200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:15.141200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:15.141200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:15.141557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:15.142085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:15.142377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:15.143991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:15.155649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 176/369] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:15.155649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:15.187943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:15.250218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09329a-ce0e-4511-99a1-ee8f6f367761 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:15.312532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:15.313053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:15.313325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:15.313592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:15.365264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342793f-ae0e-4fdb-9323-e1035add6825 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:15.392807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:15.394397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:15.424094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:15.425164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:15.425836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:15.426309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:15.427176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:15.427702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:15.429476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:15.444804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 177/370] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:15.446099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:15.457795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:15.495218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:15.533949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 194/371] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:15.534819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:15.620291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c666dc8-876b-4865-bc12-c51774328645 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:15.674056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:15.674897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:15.675329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:15.675777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:15.742603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e157061-dfe8-4ddb-9b23-3174f8bb2a0a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:15.785086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:15.786857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:15.815511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:15.816524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:15.816810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:15.817083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] PATH_INFO: `/v3/users/1266ef9190f642b7870876d39581a05c` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:15.820435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] RBAC: Authorizing `identity:get_user(user_id=1266ef9190f642b7870876d39581a05c)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:15.822442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:15.824075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 195/372] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:31:15 2022] GET /identity/v3/users/1266ef9190f642b7870876d39581a05c => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:15.824842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:15.842328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:15.909781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:15.943695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 178/373] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:15.944637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.039148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473582d-375d-49c0-8b0e-27a3fe5cbde1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.056655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdfd4-2831-4546-b43a-db881dead574 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.079578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.080204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.080582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.080856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.082924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.084491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.086567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 196/374] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:15 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:16.087928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.088637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:16.098855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.099460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.099768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.100049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.101265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] RBAC: Adding query filter params (role.id=9a827b3ce4b243f494208e3f97029a8d, user.id=1266ef9190f642b7870876d39581a05c, scope.system=all) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:16.101579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.103135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970f1a-cec3-4b5d-8702-22215f75fed7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.115785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 179/375] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 00:31:15 2022] GET /identity/v3/role_assignments?user.id=1266ef9190f642b7870876d39581a05c&scope.system=all&role.id=9a827b3ce4b243f494208e3f97029a8d => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:16.117024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f6525-a36e-4659-ad3f-411cba0b9a83 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.117309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f6525-a36e-4659-ad3f-411cba0b9a83 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.117581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f6525-a36e-4659-ad3f-411cba0b9a83 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.118834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 180/376] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:16.119577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f6525-a36e-4659-ad3f-411cba0b9a83 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.247231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f6525-a36e-4659-ad3f-411cba0b9a83 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.262667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:16.269644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a294f-11cf-4ba5-87d1-30c4a2da759b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.296569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d582a3-c14d-4055-a12c-07c0410680dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.297767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d582a3-c14d-4055-a12c-07c0410680dd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.298781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d582a3-c14d-4055-a12c-07c0410680dd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.299266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d582a3-c14d-4055-a12c-07c0410680dd None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.300704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d582a3-c14d-4055-a12c-07c0410680dd None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.302883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d582a3-c14d-4055-a12c-07c0410680dd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.305574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 181/377] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:16.307660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.308171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.308659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.332747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:16.334410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:16.336215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.337406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.337689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.337959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] PATH_INFO: `/v3/projects/e1704ba0a17f4363ab56f84c09fdea08` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.339367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] RBAC: Authorizing `identity:get_project(project_id=e1704ba0a17f4363ab56f84c09fdea08)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.340952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.342804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 197/378] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/projects/e1704ba0a17f4363ab56f84c09fdea08 => generated 344 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:16.343390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.406815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:16.436802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 182/379] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:16.438153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.551530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e43e1e-b0c1-4127-9c2c-46739550afd1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.603333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.604368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.604885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.605346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.606346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:16.606861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.608627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.615074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 198/380] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=1d2b7e763c90438eab518a55d7589260 => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:16.618427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93255f-c9cf-4b1e-8fb1-793b4c833033 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.661311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.662017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.662287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.662604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] PATH_INFO: `/v3/groups/fc76e03af9ca4abea4039ff666acfd54` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.671772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] RBAC: Authorizing `identity:get_group(group_id=fc76e03af9ca4abea4039ff666acfd54)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.673365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.679514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 183/381] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/groups/fc76e03af9ca4abea4039ff666acfd54 => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:16.681159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.730791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:16.794319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:16.817625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1d9e-15c4-460d-9f9b-db4c6833b394 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.853266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:16.860387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.861347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.861819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.862284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.863262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] RBAC: Adding query filter params (group.id=fc76e03af9ca4abea4039ff666acfd54, role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=e1704ba0a17f4363ab56f84c09fdea08) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:16.863813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:16.865567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:16.873943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 199/382] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/role_assignments?group.id=fc76e03af9ca4abea4039ff666acfd54&scope.project.id=e1704ba0a17f4363ab56f84c09fdea08&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:16.925458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c093-41e0-4011-8adc-d6ceae8e5cd3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:16.965949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:16.966958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:16.967434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:16.967941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:16.979730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:16.985175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:16.986132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:16.986920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:16.989566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 184/383] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:16.995467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:17.099860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:17.106929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d05ce7-6849-47dc-b6ee-02d10877fcb7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:17.144453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:17.145415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.145909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.146375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.147397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:17.147931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:17.149623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:17.159768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 200/384] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:17.168053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:17.287888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f3f18-a053-4834-818d-dd75bd3d15aa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:17.297250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:17.333651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:17.334633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.335125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.335691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.344245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:17.345726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:17.347447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 185/385] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:17 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:17.353222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:17.496527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53454bb1-f2a0-430f-bf0a-ba4b135a33d6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:17.544172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e12ce-0925-4395-b740-dad23806019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:17.544840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e12ce-0925-4395-b740-dad23806019c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.545120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e12ce-0925-4395-b740-dad23806019c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.545393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e12ce-0925-4395-b740-dad23806019c None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.547899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e12ce-0925-4395-b740-dad23806019c None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:17.549411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e12ce-0925-4395-b740-dad23806019c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:17.551216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 201/386] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:17 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:17.557816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:17.619792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd8179-fdc6-4e85-aba6-61adb27c754e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.620069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd8179-fdc6-4e85-aba6-61adb27c754e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.620253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd8179-fdc6-4e85-aba6-61adb27c754e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.621601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 202/387] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:17.632819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749018d1-4d2a-4052-807b-ca1399781986 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.633091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749018d1-4d2a-4052-807b-ca1399781986 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.633669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749018d1-4d2a-4052-807b-ca1399781986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.677446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-749018d1-4d2a-4052-807b-ca1399781986 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:17.680037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-749018d1-4d2a-4052-807b-ca1399781986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:17.701906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:17.725188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7ca24-2a33-41b2-b423-d45a93d6b5ed None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:17.773680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 203/388] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:17.775402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ad08-5663-4e58-85c2-45a70f5fc122 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.775915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ad08-5663-4e58-85c2-45a70f5fc122 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.776385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ad08-5663-4e58-85c2-45a70f5fc122 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.777805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:17.778119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 204/389] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:17.782152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.782152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.782152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/groups/d889e86001d94f92881e9fb8f06eed57/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.782152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] RBAC: Authorizing `identity:create_grant(project_id=74b2ffbd61a140daaf31e00fa35ab0b2, group_id=d889e86001d94f92881e9fb8f06eed57, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:17.782990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:17.784367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:17.784850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:17.785313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:17.807508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 186/390] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 00:31:17 2022] PUT /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/groups/d889e86001d94f92881e9fb8f06eed57/roles/87f1831257824560813f954348534540 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:17.811466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:17.812695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:17.905380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 205/391] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:18.198566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:18.231920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:18.240836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:18.378028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b717b49-6a72-4c25-a7fd-6a5ccf17e4aa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:18.403730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:18.436075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac9d3e-3457-489f-bf47-f1874ee1a940 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:18.458074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:18.458646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:18.459256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:18.459256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:18.485379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:18.491737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:18.492616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:18.494038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:18.497210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:18.498130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:18.498604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:18.499081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:18.508120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 187/392] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:18.513456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:18.523948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:18.534425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:18.535330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:18.536159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:18.539159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 206/393] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:18.551106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:18.678139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd75093-cc70-4895-94a4-da9e6233f13e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:18.730934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:18.731461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:18.731780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:18.732045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:18.732714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:18.733008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:18.734370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:18.751203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 207/394] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:18.757061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:18.759825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:18.766182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0229d-b317-4bd8-8a59-fc65f193339d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:18.772938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:18.830031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:18.830693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:18.830914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:18.831181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:18.831874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:18.832173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:18.833554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:18.851379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 188/395] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:18.858170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:18.957694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7770-7ec6-4c75-9997-cd8dfed73129 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:19.024990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:19.025857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7b97-cfff-4664-898e-1b6d06d88175 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:19.026035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.026441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.026841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.029011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:19.030627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:19.032461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 208/396] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:18 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:19.056634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:19.074884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:19.075549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.075828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.076099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.077215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:19.078657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:19.083538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 189/397] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:18 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:19.086187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:19.131151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:19.183889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:19.209798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d39bcb-0de3-4226-910e-a8cbc3facfd7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:19.251441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:19.252102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.252422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.252699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.254523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:19.256238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:19.259711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 209/398] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:19 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:19.264853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:19.265842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4341b1-3368-4d25-b799-9eb9548e326a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:19.309812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:19.310701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.311117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.311582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.312951 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:19.314688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:19.316698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 190/399] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:19 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:19.323367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:19.405618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a10933-cde2-4eaa-95f8-5882e65f294f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:19.446966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b173ea4-982a-42da-93c5-b599e5b7ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:19.447997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b173ea4-982a-42da-93c5-b599e5b7ecb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.448469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b173ea4-982a-42da-93c5-b599e5b7ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.448932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:19.449478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b173ea4-982a-42da-93c5-b599e5b7ecb8 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/groups/d889e86001d94f92881e9fb8f06eed57/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.453694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b173ea4-982a-42da-93c5-b599e5b7ecb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b2ffbd61a140daaf31e00fa35ab0b2, group_id=d889e86001d94f92881e9fb8f06eed57, role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:19.454807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b173ea4-982a-42da-93c5-b599e5b7ecb8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:19.460098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8a89f-c2ae-4b6f-a271-9bd21abd58c8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:19.480798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 210/400] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 00:31:19 2022] PUT /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2/groups/d889e86001d94f92881e9fb8f06eed57/roles/1d2b7e763c90438eab518a55d7589260 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:19.483112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684458b-4152-4887-be82-13568363641f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.483642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684458b-4152-4887-be82-13568363641f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.484114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684458b-4152-4887-be82-13568363641f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.486928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 211/401] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:19.493890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.495075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.495854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.504244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:19.504597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:19.505409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:19.505604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:19.505857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/groups/d889e86001d94f92881e9fb8f06eed57/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:19.508293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] RBAC: Authorizing `identity:create_grant(project_id=0718a24693104c6eada4aabe8e2ba6b6, group_id=d889e86001d94f92881e9fb8f06eed57, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:19.510422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:19.529255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:19.531281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:19.535991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 191/402] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 00:31:19 2022] PUT /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/groups/d889e86001d94f92881e9fb8f06eed57/roles/87f1831257824560813f954348534540 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:31:19.633389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 212/403] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:19.935456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:20.055127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f1926-3177-4375-bf74-85422a48d484 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:20.095757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6dad6b-a6a5-480e-b6b5-f0541e0068a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:20.096409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dad6b-a6a5-480e-b6b5-f0541e0068a7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.096706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dad6b-a6a5-480e-b6b5-f0541e0068a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.096980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dad6b-a6a5-480e-b6b5-f0541e0068a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.111387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6dad6b-a6a5-480e-b6b5-f0541e0068a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-ac6dad6b-a6a5-480e-b6b5-f0541e0068a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:20.116779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:20.117652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:20.119001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 192/404] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:20.124556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:20.242402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeefb8-47fc-4b9f-97ad-2a16be6844d9 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.242692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeefb8-47fc-4b9f-97ad-2a16be6844d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.243382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeefb8-47fc-4b9f-97ad-2a16be6844d9 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.244738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 193/405] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:20.272799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445a133-af17-45a5-88b3-452119f6fe75 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:20.277192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.277470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.277765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.307819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:20.314140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:20.314761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.315041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.315315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.316044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:20.316346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:20.317013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:20.317966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:20.318268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:20.328298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 213/406] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:20.332262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:20.339346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:20.466026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 194/407] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:20.468889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032901e-ad61-4b8e-babf-3ef1eb8cbab8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:20.505179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:20.506097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.506565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.507041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.508351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:20.510236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:20.512357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 214/408] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:20 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:20.517702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:20.657403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933132f-63d8-4d0a-87f4-6566759f8ac7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:20.692271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513244c0-f98e-41b1-ba76-702541967304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:20.692904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513244c0-f98e-41b1-ba76-702541967304 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.693180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513244c0-f98e-41b1-ba76-702541967304 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.693450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513244c0-f98e-41b1-ba76-702541967304 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.694567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513244c0-f98e-41b1-ba76-702541967304 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:20.696740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513244c0-f98e-41b1-ba76-702541967304 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:20.698691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 195/409] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:20 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:20.705468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:20.767628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc108b7-22f9-4b28-ac5b-859fef9c0425 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.767917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc108b7-22f9-4b28-ac5b-859fef9c0425 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.768102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc108b7-22f9-4b28-ac5b-859fef9c0425 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.769779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 196/410] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:20.775415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.775729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.776009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.794187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:20.795943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:20.799104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:20.806601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:20.871274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256ad0-eb86-421f-9833-fad1a24bb8e7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:20.887847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 197/411] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:20.888450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:20.927214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:20.927889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:20.928161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:20.928431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/groups/d889e86001d94f92881e9fb8f06eed57/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:20.934954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] RBAC: Authorizing `identity:create_grant(project_id=0718a24693104c6eada4aabe8e2ba6b6, group_id=d889e86001d94f92881e9fb8f06eed57, role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:20.936424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:20.960274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 215/412] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 00:31:20 2022] PUT /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6/groups/d889e86001d94f92881e9fb8f06eed57/roles/1d2b7e763c90438eab518a55d7589260 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:21.046806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e1d7-1f60-434e-8cd3-d9b17676fc9f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.100371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.101017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.101290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.101559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.115674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:21.124552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:21.132389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:21.133362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:21.135429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 198/413] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:21.140291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.209006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.278221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca1c4a-ce4c-40a5-847e-b64756d5c489 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.312076 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:21.325103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.326183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.326629 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.327057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.327971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:21.328273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:21.330170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:21.341847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 216/414] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:21.342004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74d8f7-cd98-4f3d-89a2-929a1ddd139d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.347183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.380545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.381247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.381544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.381797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.396761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:21.403046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:21.404019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:21.404819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:21.405833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 199/415] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:21.412452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.452583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa7f37-6662-47b3-8d6b-4c89061637f3 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.484304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:21.488940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.489568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.489939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.490213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] PATH_INFO: `/v3/groups/fc76e03af9ca4abea4039ff666acfd54` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.492030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] RBAC: Authorizing `identity:get_group(group_id=fc76e03af9ca4abea4039ff666acfd54)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:21.494121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:21.496194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 217/416] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:21 2022] GET /identity/v3/groups/fc76e03af9ca4abea4039ff666acfd54 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:21.501427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.528259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc91aae-cf24-40bf-a573-ada83981f31b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.554557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:21.569835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.570377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.570656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.571435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.572134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:21.572437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:21.573821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:21.583630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 200/417] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:21.589895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.617315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce924b8-053f-48d4-b55a-d5964e67b660 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.672210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.673139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.673636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.674044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] PATH_INFO: `/v3/projects/e1704ba0a17f4363ab56f84c09fdea08` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.675361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] RBAC: Authorizing `identity:get_project(project_id=e1704ba0a17f4363ab56f84c09fdea08)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:21.679504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:21.679504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 218/418] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:21 2022] GET /identity/v3/projects/e1704ba0a17f4363ab56f84c09fdea08 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:21.684945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:21.738033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472168d3-9c07-4f5b-b7d4-7d7d8bfbb40d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.787841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:21.818766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894840c-b186-4851-8809-3cbf89f807fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.819386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894840c-b186-4851-8809-3cbf89f807fc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.819694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894840c-b186-4851-8809-3cbf89f807fc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.820017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894840c-b186-4851-8809-3cbf89f807fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.847099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e66a3c-e3ee-4347-aa8e-48aff64bf09f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:21.869995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:21.887903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:21.888624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:21.888900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:21.889477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] PATH_INFO: `/v3/projects/e1704ba0a17f4363ab56f84c09fdea08/groups/fc76e03af9ca4abea4039ff666acfd54/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:21.891596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1704ba0a17f4363ab56f84c09fdea08, group_id=fc76e03af9ca4abea4039ff666acfd54, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:21.893128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:21.910751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 219/419] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 00:31:21 2022] PUT /identity/v3/projects/e1704ba0a17f4363ab56f84c09fdea08/groups/fc76e03af9ca4abea4039ff666acfd54/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:21.992001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894840c-b186-4851-8809-3cbf89f807fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:21.993569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894840c-b186-4851-8809-3cbf89f807fc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:22.119753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 201/420] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:22.128698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:22.249020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7542af-5fc5-472a-b9bd-316d6aeefb56 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:22.291344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:22.292023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.292306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.292799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.293811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:22.294262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb5be-b00d-4b17-85ec-2f76645b6128 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.294545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb5be-b00d-4b17-85ec-2f76645b6128 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.294817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb5be-b00d-4b17-85ec-2f76645b6128 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.295252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:22.296424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 202/421] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:22.298000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 220/422] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:22 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:22.301291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.301576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.301844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.303333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:22.340531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:22.341762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:22.390317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:22.395810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:22.445264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 203/423] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:22.476624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962f595-82ba-4b27-81ef-05aa6a66ad34 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:22.530281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25e24f-c58b-442b-a8fd-c87ffc763d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:22.530894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e24f-c58b-442b-a8fd-c87ffc763d1f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.531172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e24f-c58b-442b-a8fd-c87ffc763d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.531443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e24f-c58b-442b-a8fd-c87ffc763d1f None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.532568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25e24f-c58b-442b-a8fd-c87ffc763d1f None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:22.534058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25e24f-c58b-442b-a8fd-c87ffc763d1f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:22.535951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 221/424] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:22 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:22.542173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:22.653375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9507c6-13ce-4fcd-8376-a9545de2114c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:22.682945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fb594-73a2-4225-a5cc-d626f8628404 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.683221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fb594-73a2-4225-a5cc-d626f8628404 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.683503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fb594-73a2-4225-a5cc-d626f8628404 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.684841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 222/425] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:22.692216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:22.692822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.693162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.693656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.694770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:22.695041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:22.695304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:22.702029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=87f1831257824560813f954348534540, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:22.702344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:22.703818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:22.717597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 204/426] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:22 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=87f1831257824560813f954348534540 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:22.719758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:22.721023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:22.828802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 223/427] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:22.877377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:22.902125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.001226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e586e7-f470-4337-bb30-cbbc12050460 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.040437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.041068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.041347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.041617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.052092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:23.060425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:23.061337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:23.062120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 205/428] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:23.069187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.117835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.141629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.185966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f96bd-5d85-49ae-a4b7-f7597a3555f3 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.234125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.235088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.235588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.236050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.236904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] RBAC: Adding query filter params (name=member) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:23.237392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:23.238948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:23.241215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.250287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 224/429] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:23.256334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.271597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e043f-a96c-4dbb-80e3-5c0834f96aa9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.319763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.320400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.320679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.322118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.337954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:23.344280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:23.345173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:23.346025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:23.350695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 206/430] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:23.356667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.379405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97de767-ab63-420c-acef-920ec687d224 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.392936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.433914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8634a-c963-490f-9730-a682468764ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.434526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8634a-c963-490f-9730-a682468764ea None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.434806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8634a-c963-490f-9730-a682468764ea None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.435078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8634a-c963-490f-9730-a682468764ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.472378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205cfa2-0f9f-4684-b9af-9045ee128196 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.514707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.515676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.516159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.516614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.517470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:23.517960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:23.520657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:23.525104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.533252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 207/431] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:23.540137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.543557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8634a-c963-490f-9730-a682468764ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:23.546031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8634a-c963-490f-9730-a682468764ea None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:23.686324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a3b7-11bf-4906-80f0-5e214db2a809 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.703611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 225/432] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:23.711274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8634a-c963-490f-9730-a682468764ea None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.725980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.744790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9758d-78c5-4a33-90ae-5a34346dce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.745409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9758d-78c5-4a33-90ae-5a34346dce03 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.745732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9758d-78c5-4a33-90ae-5a34346dce03 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.746007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9758d-78c5-4a33-90ae-5a34346dce03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.813843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.817876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9758d-78c5-4a33-90ae-5a34346dce03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:23.819755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9758d-78c5-4a33-90ae-5a34346dce03 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:23.828094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8634a-c963-490f-9730-a682468764ea None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:23.867321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22010bcb-7e02-4284-b806-69a24bd13305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:23.868518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22010bcb-7e02-4284-b806-69a24bd13305 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.868795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22010bcb-7e02-4284-b806-69a24bd13305 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.869095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22010bcb-7e02-4284-b806-69a24bd13305 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.870241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22010bcb-7e02-4284-b806-69a24bd13305 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:23.871729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22010bcb-7e02-4284-b806-69a24bd13305 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:23.873349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 226/433] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:23.874542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f778bba-a79b-43ac-9bb7-ddfa71252649 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.874826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f778bba-a79b-43ac-9bb7-ddfa71252649 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.875095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f778bba-a79b-43ac-9bb7-ddfa71252649 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.879596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 227/434] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:23.881782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f778bba-a79b-43ac-9bb7-ddfa71252649 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:23.932820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 208/435] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:23.938412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:23.938690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:23.938958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:23.957048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:23.958220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:23.963335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:23.987512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f778bba-a79b-43ac-9bb7-ddfa71252649 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.023564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.024951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.025227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.025901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.027378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.029022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.029908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 209/436] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:24.035257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 228/437] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:23 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:24.036101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.036229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.070629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:24.148023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bf570-9533-4910-b655-08e2368bb863 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.166940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fedb4e-d5be-444b-a777-46337adc6c3d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.188973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.189940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.190407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.190867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] PATH_INFO: `/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.192193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] RBAC: Authorizing `identity:get_project(project_id=74b2ffbd61a140daaf31e00fa35ab0b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.193817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.197652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 210/438] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/projects/74b2ffbd61a140daaf31e00fa35ab0b2 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:24.203523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.205677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.206350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.206695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.207035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.208145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=87f1831257824560813f954348534540, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:24.209596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.211187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.223533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 229/439] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=87f1831257824560813f954348534540 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:24.309843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.326870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:24.328112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:24.338364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd684f-2acf-4e43-be32-6b7acf1e15e9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.380699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.381211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.381486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.381756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.382991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.384903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.386574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 211/440] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:24.392798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.463959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b8c21-4ae4-459f-bfbd-b7859f6ce364 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.505195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ad658-cc0f-412e-8e96-6ee2036666f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.505817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad658-cc0f-412e-8e96-6ee2036666f5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.506098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad658-cc0f-412e-8e96-6ee2036666f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.506366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad658-cc0f-412e-8e96-6ee2036666f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.513079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00170aaf-19a9-4b55-b426-05f474aa6ddb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.516753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ad658-cc0f-412e-8e96-6ee2036666f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-988ad658-cc0f-412e-8e96-6ee2036666f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:24.522010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:24.522926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:24.523752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:24.527615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 230/441] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:24.529023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e8c1f-ff80-49ff-bc4d-939c7fcd4499 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.529301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e8c1f-ff80-49ff-bc4d-939c7fcd4499 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.529571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e8c1f-ff80-49ff-bc4d-939c7fcd4499 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.530818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 231/442] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:24.531519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e8c1f-ff80-49ff-bc4d-939c7fcd4499 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.548077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.548676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.548953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.549227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.549952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:24.550257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.551783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f23a68-fb62-4094-9ee7-45a340cff774 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.557955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 212/443] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=74b2ffbd61a140daaf31e00fa35ab0b2&role.id=1d2b7e763c90438eab518a55d7589260 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:24.559136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.559420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.559724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.583668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:24.584820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:24.608454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:24.611618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:24.653958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e8c1f-ff80-49ff-bc4d-939c7fcd4499 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.679842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 213/444] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:24.693471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.694095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.694375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.694647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.695351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:24.695687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.697331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.712166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 232/445] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:24.713473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:24.815090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce717b51-1fae-4c21-9e28-164f70a0dbed None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:24.852400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:24.852936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:24.853209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:24.853473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:24.882360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:24.926429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:24.927985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:24.939255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.034722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 214/446] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:25.040533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.068608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd545cb-fee5-4bf7-9028-561744ad36c2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.105938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:25.106100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:25.113453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.114260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.114686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.114948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.127311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:25.132478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:25.133987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:25.134780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 233/447] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:25.140273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.151844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e511a-31ae-4c95-8e7d-6827f41f0f72 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.183847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.184461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.184732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.184998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] PATH_INFO: `/v3/projects/0718a24693104c6eada4aabe8e2ba6b6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.186102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] RBAC: Authorizing `identity:get_project(project_id=0718a24693104c6eada4aabe8e2ba6b6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.187610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.189204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 215/448] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/projects/0718a24693104c6eada4aabe8e2ba6b6 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:25.194481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.235659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbfb9c-6e4f-4aa8-80db-62f060743d7c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.269111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.269793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.270107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.270491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.271220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:25.271586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.273105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.281514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 234/449] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:25.286922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.294454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99a099-3b8b-40b3-b408-b9764a916b92 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.319203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:25.328271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.328898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.329173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.329445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] PATH_INFO: `/v3/groups/d889e86001d94f92881e9fb8f06eed57` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.330567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] RBAC: Authorizing `identity:get_group(group_id=d889e86001d94f92881e9fb8f06eed57)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.332069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.333622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 216/450] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/groups/d889e86001d94f92881e9fb8f06eed57 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:25.339822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.339960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:25.389877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6cd9-72c8-42e8-ae43-82fee11f715f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.421664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.422250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.422520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.422787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.436246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbc68-3a47-4473-a4ae-2bcf23225fa6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.469848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.470432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.470708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.470977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.471723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] RBAC: Adding query filter params (group.id=d889e86001d94f92881e9fb8f06eed57, role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=0718a24693104c6eada4aabe8e2ba6b6) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:25.472028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.473458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.479023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 217/451] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/role_assignments?group.id=d889e86001d94f92881e9fb8f06eed57&scope.project.id=0718a24693104c6eada4aabe8e2ba6b6&role.id=1d2b7e763c90438eab518a55d7589260 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:25.489032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.490536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.546249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:25.594222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 235/452] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 00:31:25.600204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.661100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:25.705472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889a91-86e2-4aa4-b042-4bf0ed929ff0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.740449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.741149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.741342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.741610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] PATH_INFO: `/v3/projects/e1704ba0a17f4363ab56f84c09fdea08` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.742778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] RBAC: Authorizing `identity:get_project(project_id=e1704ba0a17f4363ab56f84c09fdea08)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.744235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.745843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 218/453] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/projects/e1704ba0a17f4363ab56f84c09fdea08 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:25.751383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.851444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82570d2a-8537-4a23-b656-1915649efa56 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:25.882905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232fb47-2bf5-475a-bd38-b19df594c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:25.884077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232fb47-2bf5-475a-bd38-b19df594c02b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:25.884349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232fb47-2bf5-475a-bd38-b19df594c02b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:25.884610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232fb47-2bf5-475a-bd38-b19df594c02b None admin] PATH_INFO: `/v3/groups/fc76e03af9ca4abea4039ff666acfd54` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:25.885693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232fb47-2bf5-475a-bd38-b19df594c02b None admin] RBAC: Authorizing `identity:get_group(group_id=fc76e03af9ca4abea4039ff666acfd54)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:25.887107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232fb47-2bf5-475a-bd38-b19df594c02b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:25.888694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 236/454] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/groups/fc76e03af9ca4abea4039ff666acfd54 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:25.894509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:25.989984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9a1a3-1483-49ff-aef0-beccf8a3597f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:26.014114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:26.022629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:26.023214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:26.023509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:26.023786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:26.025042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] RBAC: Adding query filter params (group.id=fc76e03af9ca4abea4039ff666acfd54, role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=e1704ba0a17f4363ab56f84c09fdea08) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:26.025347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:26.026772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7d2a-154f-45b4-807f-7d71c7de1228 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:26.032291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 219/455] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 00:31:25 2022] GET /identity/v3/role_assignments?group.id=fc76e03af9ca4abea4039ff666acfd54&scope.project.id=e1704ba0a17f4363ab56f84c09fdea08&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:28.587670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c413c91-6089-4372-adf2-dcd5a5df712a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:28.588214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c413c91-6089-4372-adf2-dcd5a5df712a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:28.588214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c413c91-6089-4372-adf2-dcd5a5df712a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:28.589566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 237/456] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:28.596380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0937fc9-2830-4d36-9c4a-9e070c7b3bfc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:28.596800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0937fc9-2830-4d36-9c4a-9e070c7b3bfc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:28.596971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0937fc9-2830-4d36-9c4a-9e070c7b3bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:28.624485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c0937fc9-2830-4d36-9c4a-9e070c7b3bfc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:28.625696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c0937fc9-2830-4d36-9c4a-9e070c7b3bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:28.726189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 220/457] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:28.908618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5ad35-2877-43d1-a46a-16fbb9e4f639 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:28.908800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5ad35-2877-43d1-a46a-16fbb9e4f639 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:28.909074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5ad35-2877-43d1-a46a-16fbb9e4f639 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:28.910665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 238/458] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:28.915736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a5ec6-159f-4d8c-ba5e-9dab40203e1e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:28.916025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a5ec6-159f-4d8c-ba5e-9dab40203e1e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:28.916297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a5ec6-159f-4d8c-ba5e-9dab40203e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:28.940412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1a3a5ec6-159f-4d8c-ba5e-9dab40203e1e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:28.941603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1a3a5ec6-159f-4d8c-ba5e-9dab40203e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:29.045665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 221/459] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:29.115703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5ad35-2877-43d1-a46a-16fbb9e4f639 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.151007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:29.249189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fae9bb-db0e-41fc-9bc9-f56714d18f81 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.249479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fae9bb-db0e-41fc-9bc9-f56714d18f81 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.249657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fae9bb-db0e-41fc-9bc9-f56714d18f81 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.251173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 222/460] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:29.255957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5ad35-2877-43d1-a46a-16fbb9e4f639 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:29.262499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.262499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.262499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.293014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:29.294508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:29.324983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:29.326015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.326484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.326960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.344383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:29.347011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:29.378991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:29.380176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:29.381011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:29.381011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:29.381011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:29.382937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 239/461] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:29.392074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:29.393100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.423511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 223/462] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:29.423511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.483779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:29.522012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3366d-1692-4f32-a5be-246c789bca21 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:29.551713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d46ada-14cc-4020-aaca-fc27de386c11 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:29.563892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:29.564495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.565188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.565832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:29.566131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:29.567717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686f95a-cd30-4ad9-9043-5683d681c7ef None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:29.582460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 240/463] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:29.583869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea727c-8ffc-4957-8679-66b888758b94 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.584153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea727c-8ffc-4957-8679-66b888758b94 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.584422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea727c-8ffc-4957-8679-66b888758b94 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.585915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 241/464] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:29.588255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea727c-8ffc-4957-8679-66b888758b94 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.597676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:29.598306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.598579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.598847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.611664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:29.613148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-336a1d69-5d39-4b3f-a37e-b2a67430f29b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:29.632195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:29.634785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 224/465] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.636624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.661996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:29.663538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:29.693496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:29.722447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea727c-8ffc-4957-8679-66b888758b94 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:29.738742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 225/466] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:29.739908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.765838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:29.772419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:29.773096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.773447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.774379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.775082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:29.776689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:29.803461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 242/467] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:29 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:29.804349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.860067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57a3e7-8d44-4fe8-bd2d-be1c77b29c83 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:29.899392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:29.900447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:29.900923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:29.901406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:29.902276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:29.902810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:29.904511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:29.918017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 226/468] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:29.926115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:29.933615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930722be-639f-4c28-a21f-814cfd68807a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:29.979174 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:30.016861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.017512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.017796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.018059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.046431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.048217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:30.076599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:30.077568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:30.078430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:30.078430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:30.078430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:30.078430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 243/469] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:30.085047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:30.104253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f0dc-5f5b-4d69-8fd6-9b64f0d2584a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:30.136095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:30.146555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.147393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.147867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.148284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.150142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.152118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.178998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 227/470] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:29 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:30.180164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:30.200663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffd32c-d0e9-4173-8ba8-c9963d8bda44 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:30.257513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.258955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.259237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.259531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.278609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.280467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.293221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-0f76c616-e2e6-4b34-9f5b-36d909f9678e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:30.306111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:30.307227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:30.308177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:30.308177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:30.308177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:30.308177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 244/471] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:30.311505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4d3f9-d7d3-4f6a-b414-6fe91d92ff1f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.311505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4d3f9-d7d3-4f6a-b414-6fe91d92ff1f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.311505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4d3f9-d7d3-4f6a-b414-6fe91d92ff1f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.311505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 245/472] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:30.313723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4d3f9-d7d3-4f6a-b414-6fe91d92ff1f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:30.329471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a41043-507e-4266-9052-579a31444a6a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:30.375329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.375984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.376264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.376535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.377202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:30.377487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.378865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594b988-041e-4552-8e08-b245138fcc52 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.392191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 228/473] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:30.393452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.393739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.394016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.417385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:30.418606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:30.441563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:30.455695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4d3f9-d7d3-4f6a-b414-6fe91d92ff1f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:30.491744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.492559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.493889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.493889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.494384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:30.494623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.496110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.504240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 229/474] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:30.504901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:30.512399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 246/475] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:30.517376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:30.594987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:30.630575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d7971-a8ee-4b36-99f9-ffe9b17de5ce None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:30.638204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e584c-c5c3-4aba-8b8a-b0143d508801 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:30.670296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.671166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.671569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.673778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.674637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.676285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.683424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:30.684254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:30.684676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:30.685092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:30.686141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:30.688788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:30.703823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 247/476] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:30 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:30.716146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 230/477] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:30 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:30.992073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:31.005218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:31.108120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e1de7-aade-4a18-b97a-2b18fc44357f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:31.151732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:31.152656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:31.152932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:31.153197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:31.153884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:31.155255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:31.174720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.sql.core [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 00:31:31.174720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 00:31:31.174720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [parameters: {'id': 'd7764871f94a4234981fd04f404aa585', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 00:31:31.174720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81609) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application conn.query(q) Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result.read() Sep 15 00:31:31.190824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return ref.to_dict() Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 00:31:31.191755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application next(self.gen) Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application yield resource Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application next(self.gen) Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise self.value Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application session.commit() Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 00:31:31.192683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.session.flush() Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application compat.raise_( Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 00:31:31.193538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 00:31:31.194375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application conn.query(q) Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result.read() Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 00:31:31.195175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [parameters: {'id': 'd7764871f94a4234981fd04f404aa585', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:31.196767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 00:31:31.197566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:31.198409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 248/478] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 00:31:30 2022] POST /identity/v3/roles => generated 144 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:31.204325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:31.369817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa045902-a5a8-4d21-963b-26ee6d3c6b2c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:31.401063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:31.417076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b9556-f20c-4aa0-a82d-7e447cdb2e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:31.417962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b9556-f20c-4aa0-a82d-7e447cdb2e3a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:31.418493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b9556-f20c-4aa0-a82d-7e447cdb2e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:31.418908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b9556-f20c-4aa0-a82d-7e447cdb2e3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:31.438148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b9556-f20c-4aa0-a82d-7e447cdb2e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-082b9556-f20c-4aa0-a82d-7e447cdb2e3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:31.443761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:31.444875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 00:31:31.446444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:31.448573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 231/479] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:31:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:31.456559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:31.572910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:31.618755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9c89-f0c2-4f21-b459-c19a34e4e2de None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:31.668283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:31.668879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:31.669088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:31.669355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:31.670032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:31.670332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:31.671738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b7765-ea8d-401e-a0ba-b31ad4f8926c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:31.682505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 249/480] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:32.708788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db824ad3-eead-48f0-85f3-593827bd832f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:32.709737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db824ad3-eead-48f0-85f3-593827bd832f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:32.710217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db824ad3-eead-48f0-85f3-593827bd832f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:32.712627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 232/481] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:32.718271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:32.719066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:32.719603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:32.744510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69bdb-1e96-421a-a91f-776ae744b15e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:32.745203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69bdb-1e96-421a-a91f-776ae744b15e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:32.745714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69bdb-1e96-421a-a91f-776ae744b15e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:32.746018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:32.747224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:32.747880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 233/482] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:32.753325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:32.753597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:32.753864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:32.785062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:32.787976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:32.864947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 250/483] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:32.888823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 234/484] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:33.213812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.292148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:33.302679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.335785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76907ccd-3b4c-414b-887f-345328da6722 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:33.372438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa659-77d3-4b59-912c-8e70ade4aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:33.373078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa659-77d3-4b59-912c-8e70ade4aeff None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.373355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa659-77d3-4b59-912c-8e70ade4aeff None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.373625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa659-77d3-4b59-912c-8e70ade4aeff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.389058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aa659-77d3-4b59-912c-8e70ade4aeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ac7aa659-77d3-4b59-912c-8e70ade4aeff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:33.394312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:33.395206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:33.396118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 251/485] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:33.401548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb2f7f-18df-4eab-b37b-701de9e86861 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.401899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb2f7f-18df-4eab-b37b-701de9e86861 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.402602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb2f7f-18df-4eab-b37b-701de9e86861 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.408016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 252/486] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:33.408813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb2f7f-18df-4eab-b37b-701de9e86861 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.466775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5a5c6-5277-4d12-b3c2-9d892995ca28 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:33.487263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:33.516016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df899736-f32d-41a2-b550-ebd70a6f9956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:33.517115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df899736-f32d-41a2-b550-ebd70a6f9956 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.517539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df899736-f32d-41a2-b550-ebd70a6f9956 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.518135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df899736-f32d-41a2-b550-ebd70a6f9956 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.519246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb2f7f-18df-4eab-b37b-701de9e86861 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:33.531663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df899736-f32d-41a2-b550-ebd70a6f9956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-df899736-f32d-41a2-b550-ebd70a6f9956 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:33.538200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:33.539731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:33.540648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:33.543639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 235/487] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:33.545780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.546247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.546679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.558101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:33.569505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:33.571585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:33.571953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:33.572401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.572749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.573133 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.573875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:33.574314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:33.575978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:33.588130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 253/488] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:33.593653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.658916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 236/489] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:33.659870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.715728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6742b0d-5186-4254-838c-404a13a2cedf None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:33.757785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:33.759053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.759129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.759377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.760063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:33.760358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:33.761920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9eaec-54b1-4c7d-ac60-589ea0896cc0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:33.770159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 254/490] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:33.771343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da788553-c1a8-499f-a50d-96807a4631aa None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.771647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da788553-c1a8-499f-a50d-96807a4631aa None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.771914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da788553-c1a8-499f-a50d-96807a4631aa None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.777307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 255/491] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:33.778085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da788553-c1a8-499f-a50d-96807a4631aa None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.793646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dba034-0b73-4c11-b0cd-94de992c375c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:33.797831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:33.831772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:33.842296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:33.843823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.844299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.844975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.861776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:33.864028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8f35f8f9-cb53-4152-83e2-7ca6727dd09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:33.886031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:33.886956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:33.887807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:33.887807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:33.887807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:33.889020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da788553-c1a8-499f-a50d-96807a4631aa None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:33.889481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 237/492] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:33.891021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.892392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.892831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.913165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:33.915028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:33.934821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:33.935735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:33.936147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:33.937106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:33.949279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:33.950706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:33.967164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:33.968122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:33.968971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:33.968971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:33.968971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:33.968971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 256/493] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:33.969822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:33.991032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 238/494] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:33.996640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.078726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.083370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d5c9a-b99a-4103-a8a3-d76e451722f7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.101992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.121246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48318575-3515-4892-b3f4-cbf5aa4955e7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.137951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.139297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.140065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.140517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.141609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:34.142117 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.144112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.160722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.161487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.161829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.166958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.167073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 257/495] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:34.167073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.187778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:34.195176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:34.196144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:34.196947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:34.203948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 239/496] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:34.204720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.286268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57eb2-facc-455f-9103-77c00cb61359 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.327028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.328045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.328507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59f592-cc97-4dc6-8f7e-238642baf294 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.328720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.329226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.330306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:34.331675 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.333237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.352860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 258/497] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:34.352860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.367957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.368587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.368874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.369144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.373286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.376710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.378170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:34.391664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:34.392618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 240/498] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:34.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.407222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.456741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640935e-d624-4dfe-aa5f-8f4e12fe3e4a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.494948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.495584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.495863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.496224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.497331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:34.497635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.499015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.508961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 259/499] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:34.509768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.511263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18f58-320c-41f2-90cc-1c3252e1d7bf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.545868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3733f-a280-4971-b329-eff59232c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.546488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3733f-a280-4971-b329-eff59232c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.546760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3733f-a280-4971-b329-eff59232c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.547027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3733f-a280-4971-b329-eff59232c4a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.557328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3733f-a280-4971-b329-eff59232c4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3ac3733f-a280-4971-b329-eff59232c4a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:34.562466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:34.563425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 241/500] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:34.564771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c587f06-f232-4ef5-8fa1-80b358a33415 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.565497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c587f06-f232-4ef5-8fa1-80b358a33415 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.565497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c587f06-f232-4ef5-8fa1-80b358a33415 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.566421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 242/501] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:34.567099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c587f06-f232-4ef5-8fa1-80b358a33415 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.596823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.614879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eac8f-f033-4284-9b87-5f85bfa83da6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.631277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.646775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.647396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.647698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.647966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.655825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.657246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.669354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c587f06-f232-4ef5-8fa1-80b358a33415 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:34.669915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:34.670772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 260/502] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:34.672361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.707428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.708052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.708329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.708599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.709316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:34.709618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.711635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.746679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 243/503] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:34.747498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.769502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ea087-1fc8-4fd1-9909-0d7af57c88e8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.801716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.802328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.802601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.802867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.812845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.814275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.815983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-945ef739-a9f1-46bf-b260-1eeed1e27f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:34.829451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:34.830393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:34.831232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:34.831232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:34.831232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:34.831232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 261/504] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:34.831797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.832046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.832319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.853584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:34.853756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe937eef-239b-4248-ab42-72445ef53429 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:34.855160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:34.877814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:34.890352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:34.890952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:34.891227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:34.891518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:34.892181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:34.892479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:34.893857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:34.902142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 244/505] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:34.902912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:34.921666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 262/506] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:34.922499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.006656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d418df-2a69-4ade-b9ae-2fafa096f0ec None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.034033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fca4d-c1c4-45e6-9e7e-3987a1abec88 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.046023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.046615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.046885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.047147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.047883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:35.048184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.049609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.060342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.068084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 245/507] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:35.068874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.077332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.077936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.078207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.078472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.117880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.151113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.152980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.176561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd8633-f36f-42f4-9024-7b1798f968cc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.211963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.212563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.212835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.213106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.213740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:35.214043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.215411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.226133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 246/508] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:35.226933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.253193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 263/509] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:35.254035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.279946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.326821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156edbd4-2cb1-4c71-b6c5-9624ebfacb86 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.329069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.352860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5843af-bc38-4f1b-a4bc-009e9546e34e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.359561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.360178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.360452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.360717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.371565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.372953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.386323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.386914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.387187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:35.387440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:35.388470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:35.389479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:35.389479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:35.389479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:35.389685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.389762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 247/510] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:35.389762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.455711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.457157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.481237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.488023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a5c0f-4914-498a-ab2d-9efbbab6e9b6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.521519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.522140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.522417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.522686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.533787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.535258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:35.549101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:35.549988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:35.550821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:35.550821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:35.550821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:35.550821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 248/511] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:35.551040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.561138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.561273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 264/512] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:35.562156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.652458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd31980-a6cb-45ae-bbec-32a187c3870b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.661380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a9fc6-a4f4-428a-b0f5-47d67abfb346 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.684888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.685516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.685797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.686073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.686262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.693824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.695287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.697468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.698090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.698367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.698638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:35.706989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:35.708063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 249/513] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:35.709163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.710923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.712941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:35.727873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:35.728766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:35.729612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:35.729612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:35.729612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:35.729612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 265/514] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:35.729832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.783929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.817074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17af25f-4c4a-4249-8902-e9f087af2009 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.826966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e784767-1654-4a1f-a2cb-91f967119760 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.850651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.851239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.851541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.851842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.852472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:35.852773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.854139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.859737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:35.860330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:35.860985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:35.860985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:35.861495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:35.861797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:35.863156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:35.865931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 250/515] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:35.866722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.878447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 266/516] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:35.880424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:35.918427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:35.969233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da16df-c8dd-4960-9413-20a8b56bba38 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.981277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42b9ce-6a98-4fd3-b6e4-a798d6a48ab9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:35.995898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.003700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.004327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.004604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.004872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.014518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.014903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.015113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.015389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.015681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.016390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.016478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:36.016689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.018079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:36.031238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:36.032160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:36.032996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:36.032996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:36.032996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:36.032996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 251/517] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:36.033219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.054576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 267/518] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 15 00:31:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:36.055436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.135699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7aeb73-6c9b-42c0-b044-4d8461c77443 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.143498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.150845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80f919-7a40-4938-b662-32acc3c3aa21 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.169863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.170399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.170733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.171008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.171675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:36.171973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.173357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.183895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.184528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.184621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 252/519] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:36.184809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.185078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.185397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.198182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.201237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:36.220121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:36.221233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:36.222322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 268/520] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:36.223000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.230429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.290509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31c6af-25b5-4f77-ab5e-72dc2a6b6251 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.323344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.323968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.324239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.324502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.325126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.326554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.331094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a726-fbf5-477a-92a9-d494158d22cc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.346206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 253/521] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:36 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:36.347201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.368140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.368756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.369026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.369582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.370251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:36.370550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.372009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.393532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 269/522] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:36.393532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.411759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.459639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.466957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6950ed0-4883-413f-91d9-ff97f93490d2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.506700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.507416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.507776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.508021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.512256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df1bcf-f551-4192-b50d-4d300fa65572 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.573572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.574507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.574998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.575275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.582666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.584239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:36.596676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:36.597568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:36.598404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 270/523] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:36.598860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.599963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.603281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.686894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.711633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680da937-8ac7-4503-8f1d-bd8ee9934317 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.716356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.729404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 254/524] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:36.729404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.746359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.746968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.747247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.747541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.748272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:36.748576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.749983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.768121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 271/525] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:36.768943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.844773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa1bef-7085-4325-b8db-6f2d04f50f60 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.888104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.888763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.889007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.889308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.891530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1714b-d34d-44d8-970d-65ffaf3c5645 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:36.896436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.897897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:36.909937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:36.910988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 255/526] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:36.911869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.933081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:36.939219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:36.939854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:36.940134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:36.940399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:36.941093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:36.941400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:36.942816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:36.966083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 272/527] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:36.966083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:36.978245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.028767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354fdc8-0dfa-49ea-9b75-51d8a022a25a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.065671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.066295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.066576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.066850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.071430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb277a-e364-4ef4-bc72-99bcc315ec93 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.080046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.082193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:37.098690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:37.099678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:37.100526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:37.100526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:37.100526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:37.100526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 256/528] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:37.100739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.109913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.110532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.110823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.111078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.179025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.190359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.191848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.208040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52526fec-f2a1-4d65-8e57-517630e0463b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.218491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.246820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.247863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.248356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.248784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.249787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:37.250227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.252063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.271001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 257/529] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:37.271849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.306735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 273/530] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:37.307908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.382028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25176258-bfc6-40a5-989b-c640305b9a95 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.406857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.417219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1dd7c-1f00-4552-bf90-708ac47c2c77 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.418248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.419010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.420048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.420335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.421064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=4784d64225054ea9a987581a00b6807b) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:37.421372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.422823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.429248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 258/531] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/role_assignments?user.id=4784d64225054ea9a987581a00b6807b&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:37.430540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.467773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.469159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.469639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.470372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.472031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:37.472569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.474500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.476131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.497818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 274/532] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:37.498692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.565566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cf4f4-3d02-4fa5-8e14-929b89c62d49 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.606737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.607361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.607665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.607935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.608678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=0d2cf22448884e1f8758b9a448d001ef) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:37.608985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.611019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.623069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 259/533] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/role_assignments?user.id=0d2cf22448884e1f8758b9a448d001ef&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:37.623969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.646923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8791a-31ca-453c-b4b2-1b3d57cc42bc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.698128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.699593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.700050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.700752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.716298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.718143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:37.737284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:37.738218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:37.739068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:37.739068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:37.739068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:37.741839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 275/534] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:37.747440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.779936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.786506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:37.797719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed553ef-bef1-4961-bcca-158c909f457c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:37.845868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:37.846783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:37.847064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:37.847336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:37.856117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:37.857592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:37.870740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:37.871650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:37.872522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 260/535] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:37.879102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:37.941773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb1a05-c86a-44d4-bfd3-38249b74e159 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:38.003830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c53a2-dd7c-4050-af10-ef7739bbf53b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:38.012010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:38.012637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.012917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.013190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.013843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:38.014146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:38.015567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:38.031914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 276/536] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:38.037915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:38.075754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:38.076363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.076639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.076905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.077605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:38.077909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:38.079376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:38.111454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 261/537] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:38.118151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:38.155879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:38.176063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:38.251892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7a03d-5f7d-465f-bece-12e89938d54b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:38.272351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351125d5-0417-4528-bce4-5c75e39813aa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:38.307409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:38.309989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.310269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.310540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.318507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:38.319994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:38.333339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:38.334213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:38.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:38.335447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 277/538] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:38.341243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:38.373496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:38.374105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.374382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.374653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.375380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=24bf5028fe8d4a619897cf34ba3ff809) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:38.375713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:38.377156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:38.389704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 262/539] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:38 2022] GET /identity/v3/role_assignments?user.id=24bf5028fe8d4a619897cf34ba3ff809&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:38.468622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1adea-6f11-41ee-876e-7a81889eee53 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:38.506021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:38.527705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:38.528243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.528522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.528792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.529494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:38.529798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:38.531284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb42a-df42-4152-a843-0b269f9477c7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:38.550336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 278/540] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:38.557570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:38.657351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:38.682587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d3ac-d8e9-427c-b8c1-2d02fdc54db7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:38.738040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:38.738649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.738924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.739194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.740014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=7727b52cd17b41708b243dac7b4f2c16) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:38.740322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:38.741767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3ae4d-7a66-4a4e-8b69-33650905ea15 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:38.751950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 263/541] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:38 2022] GET /identity/v3/role_assignments?user.id=7727b52cd17b41708b243dac7b4f2c16&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:38.801014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e10e1d-b81a-453a-82f3-a41155a3896b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.801289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e10e1d-b81a-453a-82f3-a41155a3896b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.801489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e10e1d-b81a-453a-82f3-a41155a3896b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.802872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 279/542] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:38.809047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:38.809425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:38.809601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:38.829398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:38.830671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:38.927746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 264/543] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:39.284855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e10e1d-b81a-453a-82f3-a41155a3896b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:39.405018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e10e1d-b81a-453a-82f3-a41155a3896b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:39.438682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:39.439311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:39.439633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:39.439906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:39.450907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:39.455854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:39.456723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:39.457517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 280/544] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:39.463629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:39.573861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:39.612332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55d1d4-dabf-4b1c-8d6e-ec9561121b6b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:39.673429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:39.674996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:39.675542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:39.676005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:39.676871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:39.677416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:39.678968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac9ea3-1475-4cb9-95ba-ba325ba806be None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:39.694903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 265/545] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:39.701560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:39.786931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:39.916600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571c84-0ca9-4de7-8043-8c7f573282c9 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:39.916838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571c84-0ca9-4de7-8043-8c7f573282c9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:39.917270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571c84-0ca9-4de7-8043-8c7f573282c9 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:39.918806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 266/546] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:39.924504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:39.924917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:39.925252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:39.934608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4588-36bc-4024-aaf8-58e0afc68805 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:39.971515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:39.972312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:39.984456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:39.985103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:39.985381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:39.985651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.000191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:40.001654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:40.027519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:40.028485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:40.029331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:40.029331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:40.029331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:40.029331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 281/547] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:40.041031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:40.162379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 267/548] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:40.208225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af3546-7170-4dcf-b95a-657de63e2412 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:40.211804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:40.293304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:40.293922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:40.294182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:40.294804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.300526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:40.301096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:40.303237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69569b4-8dfe-43b7-9183-c850f2f0ec3f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:40.327061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 282/549] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:40.334894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:40.361764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dad577-dc22-479c-bc00-db6690e6cdcc None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:40.362056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dad577-dc22-479c-bc00-db6690e6cdcc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:40.362318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dad577-dc22-479c-bc00-db6690e6cdcc None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.365644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 283/550] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:40.369844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:40.370130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:40.370410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.419539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:40.420746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:40.511583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:40.576235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f9b58-ad3b-4f3a-8470-b09facc3bfe4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:40.576447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 284/551] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:40.577299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:40.663299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:40.665061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:40.665563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:40.666029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.677093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:40.678967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:40.695051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:40.696299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:40.697234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:40.699957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 268/552] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:40.708034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:40.751996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:40.779905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d1ffa-601a-4730-8a8c-5787875fb1d2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:40.839073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:40.839725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:40.839998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:40.840266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.855873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:40.860612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:40.861683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:40.862529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 285/553] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:40.870394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:40.888543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7655358-969e-4710-9d21-7b6960ebcfe6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:40.916202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:40.925220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:40.925922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:40.926202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:40.926465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:40.927186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:40.927508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:40.929101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:40.948046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 269/554] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 00:31:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:40.953228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.005167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ece0-88e1-4e4c-91f3-fe9a32ea9e4d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.054978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.056116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.056824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.057705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.058831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:41.059314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.060153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5e7b-a26e-41ba-b0a7-23fc63594ded None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.061072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.073170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 286/555] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:41.074239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.088887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.102619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.103284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.103586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.103857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.118557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:41.124640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:41.125555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 270/556] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:41.131684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.177036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.191297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078686-2467-4d8b-a772-5be7ddeb35f0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.227650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.228276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.228516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.228779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.232956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e56ce0-1c8a-43c3-aab6-5dc1e4831442 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.268347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.268967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.269359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.269629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.281852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.283329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-74758221-7bcd-419d-974c-f8fcdbe19e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:41.299153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:41.300576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:41.301459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:41.301459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:41.301459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:41.301459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 271/557] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:41.302206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fd979-f027-40b0-b7ab-f492bf0c39bb None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.302206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fd979-f027-40b0-b7ab-f492bf0c39bb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.302371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fd979-f027-40b0-b7ab-f492bf0c39bb None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.308326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 272/558] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:41.308326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fd979-f027-40b0-b7ab-f492bf0c39bb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.309178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.310644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.322173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.404861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.413172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fd979-f027-40b0-b7ab-f492bf0c39bb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.422915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 287/559] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:41.422915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.450749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.451353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.451763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.452003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.452606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:41.452905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.454344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63413-9ca6-428b-801e-fb55d6be90fd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.467447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 273/560] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:41.468763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.469030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.469392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.488297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:41.489475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:41.526867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dca84-6ca2-431b-99e5-f2349c0cc246 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.537650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.568929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 274/561] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:41.569856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.570787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.572119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.572552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.573435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:41.573734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.575089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.592681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 288/562] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:41.594385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.679765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.693349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb879d-6178-4a7c-987e-e72109243cf4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.710415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cec3d-1de2-40ca-8205-6e26844a18fb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.732916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.733551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.733832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.734102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.744881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.745369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.745485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.745758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.746022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.746840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.756509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.758757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:41.762012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:41.762970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:41.764014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:41.764014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:41.764014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:41.764014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 275/563] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:41.764287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-a6002a28-1595-4bc5-ae4a-ba6ec9476df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:41.778057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:41.779213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:41.780298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:41.780298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:41.780298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:41.780298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 289/564] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:41.780529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff530-3f6a-40ed-b41b-66ad147ac43b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.780640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff530-3f6a-40ed-b41b-66ad147ac43b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.780910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff530-3f6a-40ed-b41b-66ad147ac43b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.786309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 290/565] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:41.786309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ff530-3f6a-40ed-b41b-66ad147ac43b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.807036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.875574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11f16a-1583-4e00-95a9-5baa5fed9fb1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.886102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ff530-3f6a-40ed-b41b-66ad147ac43b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:41.908657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.909274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.909557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.909831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.916983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.917245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:41.917835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.918109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.918374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.918480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.919007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:41.919303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:41.920661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5344c95-96ed-4651-b04d-b5a45c448b97 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:41.920964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:41.930703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:41.931739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 276/566] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:41.932729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:41.937306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 291/567] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:41.938489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:41.938777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:41.939049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:41.957037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:41.958238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:42.035505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 292/568] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:42.035505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.035786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722b31-41f2-4eac-bf41-0f7a831dacd1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.037002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.069887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.071039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.071321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.071633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.072279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:42.072585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.073962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.086119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 277/569] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:42.088944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.126353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57806be8-7f60-4a8c-a7ed-a45f1e7c2c2d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.149934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.164186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.164809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.165082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.165346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.175312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:42.180771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:42.181694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:42.182543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 293/570] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:42.183249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.201440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4526-3950-424f-b833-1fdfbbadaf1a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.237532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.238278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.238556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.238821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.239573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:42.239870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.241301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.256562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.269577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 278/571] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:42.269577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.289422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1ad77-a0da-4eb6-8f48-958744adfb37 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.323806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.324428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.324699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.324971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.331578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.333013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:42.344880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:42.345832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:42.346733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 294/572] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:42.347323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.374761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdaac97-35a3-4fc0-b4aa-6c4eb60e7d1e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.391583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.408580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.409203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.409479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.409756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.417054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.418514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:42.430340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:42.431436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 279/573] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:42.432420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.446412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69d760-fbc7-48f3-bb3d-83761c1717b9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.473245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.480331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.480926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.481202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.481467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.482116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:42.482409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.483779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.491469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 295/574] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:42.492830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.533345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6765304-4d23-448d-94bd-640d42bc0f1e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.566724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.567342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.567661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.567936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.578462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.579964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.587734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319467a4-3519-445a-b104-cae21cd61e1a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:42.594261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:42.595369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:42.596491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:42.596491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:42.596491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:42.596491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 280/575] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:42.596491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.628196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.629121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.629536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.629947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.632751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.647156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:42.656410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:42.657460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:42.658426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 296/576] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:42.659239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.705109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50528-a214-4c83-aafd-c9f7ed6fe08e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.721574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.738340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.738940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.739213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.739504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.740198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:42.740502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.741939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.753559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 281/577] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:42.754344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.767425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba55d8b-4811-424a-9104-81e1d5073e4e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.802197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.802797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.803075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.803342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.804088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:42.804389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.805817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.822295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 297/578] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:42.823081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.844079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.855133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195d5af-ef30-4ff8-8ed5-5ec55c19635d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.888325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.888944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.889221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.889490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.899919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.901304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:42.916800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:42.917801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:42.919094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:42.919094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:42.919094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:42.919094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 282/579] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:42.920274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:42.920487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c706c-2bd0-4589-807a-9910f29c2e4e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:42.932225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:42.955304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:42.955921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:42.956192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:42.956458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:42.957085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:42.957378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:42.958706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:42.970024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 298/580] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:42.970813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.025834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d6ca2-3b26-452f-878d-c80946d70ce3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.057913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.058502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.058776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.059042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.059726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.060024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.061406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.068929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 283/581] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:43.069692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.074149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9c4e7-7a35-4919-ac1f-fe9c9dc6fef1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.075024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.107568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.108159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.108457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.108725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.109435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=0587d0fb93ad478fa931141ac9e1a5de) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.109736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.111131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.116277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 299/582] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:42 2022] GET /identity/v3/role_assignments?user.id=0587d0fb93ad478fa931141ac9e1a5de&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:43.117037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.174047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61247-d720-487c-b858-6a35fda34c78 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.177242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.212705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.213343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.213626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.213899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.225345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.226943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:43.242195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:43.243230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:43.244159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:43.244159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:43.244159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:43.244159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 284/583] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:43.245001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.267276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5ad26-394f-45d3-80d0-96e61571ebd8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.306387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.307001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.307647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.307756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.308322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.308630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.310438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.310857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.325882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 300/584] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:43.326712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.361119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104e751-c026-4be5-9629-189e31bc88b6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.397845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.398529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.398779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.399048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.406222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.407818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.419502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:43.420889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:43.422018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:43.422965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 285/585] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:43.423908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.433506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3121-e72a-4086-b147-75c2823fc7f6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.469021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.469638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.469909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.470235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.480571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.481990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:43.496832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:43.497770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:43.498672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:43.498672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:43.498672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:43.498672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 301/586] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:43.498672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.532058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627c25b-fbd1-4c52-b6fe-8bdf6ae41bac None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.549568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.569399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.570012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.570324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.570593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.571233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.571563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.572953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.584746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 286/587] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:43.585564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.613520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb339e-1c0e-4dcd-842e-1e2757617fa1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.648896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.649665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.650010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.650354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.658347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.659826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:43.671692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:43.672663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:43.673611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 302/588] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:43.674308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.685548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.704942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382cfe-0868-4685-8c75-6daaed41eac1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.740795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.741400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.741678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.742000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.742704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.743014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.744462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.761361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 287/589] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:43.761361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.788096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d3eb8-2f48-4d17-a532-b498ce1d9f28 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.799072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.824411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.825202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.825662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.825933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.826582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.826921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.828451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.846963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 303/590] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:43.846963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.870468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67258a-a6ae-435d-8054-f6b06335edef None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.906223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.906938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.907286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.907665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.912413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:43.914779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.916405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:43.928118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:43.929214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:43.930330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:43.934958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 288/591] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:43.935680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:43.951618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a81ee-3565-422f-a792-ded171482e3c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:43.989366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:43.990135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:43.990422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:43.990751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:43.991565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:43.991980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:43.993644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.017219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 304/592] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 15 00:31:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:44.017219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.034907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:44.042188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e5e39-b73c-42f3-92ad-497a481bd810 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.076708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.077358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.077638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.077908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/4784d64225054ea9a987581a00b6807b/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.093110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=4784d64225054ea9a987581a00b6807b, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.094809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.107565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 289/593] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:31:43 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/4784d64225054ea9a987581a00b6807b/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:44.108526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.130971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ea83b-59b1-4501-9393-5ba8f6f22510 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.159233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:44.175362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.176521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.177030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.177528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.187925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.190339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:44.205881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:44.206929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:31:44.207890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:44.210724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 305/594] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:44.211902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.244515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfa83a-b64a-468b-8e32-4522c5e6b039 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.295938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.296958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.297233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.297505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.299132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:44.299442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.300919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.316223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 290/595] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:44.317100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.336455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:44.351003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2387948-3161-4b12-a481-c770434bafd5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.389136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.389777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.390142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.390418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.402185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.403803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:44.420059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:44.421124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:44.422041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:44.422041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:44.422041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:44.422041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 306/596] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:44.422276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.457029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce8e4c-cca0-44b9-8154-24bd4f1418b4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.462830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:44.500899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.501872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.503033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.503473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.504405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:44.504864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.506501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.530139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 291/597] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:44.531197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.536033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aabda-0ab2-4ec8-a067-2901609c7bfc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.573642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.574448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.574823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.575238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/0d2cf22448884e1f8758b9a448d001ef/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.591286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=0d2cf22448884e1f8758b9a448d001ef, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.593036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.608852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:44.612709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 307/598] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:31:44 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/0d2cf22448884e1f8758b9a448d001ef/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:44.614315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.675982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23217dc2-6827-4c6d-8293-d953ba5ad607 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.732885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:44.756462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176f80e-2922-4ef9-b875-fbba4328a74f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:44.761385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.761678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.761884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.762261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.762984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:44.763294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.764786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.790621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 292/599] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:44.795893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.802546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:44.803471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:44.803977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:44.806067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:44.818937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:44.821605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:44.841352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:44.842355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:44.843236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:44.843236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:44.843236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:44.846365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 308/600] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:44.854017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:44.959029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8684d-63d1-4261-bc70-9339625e0ccf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.003932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:45.017358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81b803-bd05-4f13-a6b8-ce20179faacf None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.028469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.029211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.029489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.029760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.046317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.047917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.053175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.053782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.054355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.054652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.055294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:45.055621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.056965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.064291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:45.068797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:45.069800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:45.070760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:45.074756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 293/602] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:45.074972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 309/602] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:45.075471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:45.079066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:45.192990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364076c7-15cf-41f8-bc44-e891fa42c411 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.233239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.233853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.234131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.234398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.235087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:45.235384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.237139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.250546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 310/603] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:45.255807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:45.330458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08675cda-f9a1-4df3-88c8-0684932cbb62 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.373110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:45.421017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ab69-b985-41d7-b14a-b9b45fe53775 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.438634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.439275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.439617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.439890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.455674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.456720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076e4eb-7827-43c4-ab0d-0e93983d13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.457384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076e4eb-7827-43c4-ab0d-0e93983d13e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.457558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.457655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076e4eb-7827-43c4-ab0d-0e93983d13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.457909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076e4eb-7827-43c4-ab0d-0e93983d13e8 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/24bf5028fe8d4a619897cf34ba3ff809/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8764a6d0-1bad-4dc0-8e1c-3ee35cba2e35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:45.469653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:45.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:45.471902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:45.472583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 294/604] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:45.473820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435561c-a382-4d78-8240-da1fd013eb44 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.474169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435561c-a382-4d78-8240-da1fd013eb44 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.474504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435561c-a382-4d78-8240-da1fd013eb44 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.476124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 295/605] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:45.478943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076e4eb-7827-43c4-ab0d-0e93983d13e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=24bf5028fe8d4a619897cf34ba3ff809, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.485119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076e4eb-7827-43c4-ab0d-0e93983d13e8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.485272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435561c-a382-4d78-8240-da1fd013eb44 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:45.499355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 311/606] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:31:45 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/24bf5028fe8d4a619897cf34ba3ff809/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:31:45.501312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.501663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.501997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.527566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:45.529274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:45.548450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:45.637974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435561c-a382-4d78-8240-da1fd013eb44 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.641886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 312/607] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:45.698331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.698996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.699342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.699651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.700354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:45.700662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.702113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.722488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 296/608] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:45.729297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:45.776431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:45.841012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b020c7-72ad-4a73-a6c5-a2b3074d5a9d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:45.875986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:45.876714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:45.876994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:45.877315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/7727b52cd17b41708b243dac7b4f2c16/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:45.893582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=7727b52cd17b41708b243dac7b4f2c16, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:45.895074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:45.912026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 313/609] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:31:45 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/7727b52cd17b41708b243dac7b4f2c16/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:46.003615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:46.141275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3fe7c-5a5a-4446-b2dc-2b5862dee613 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:46.149295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:46.186066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:46.186703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:46.186976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:46.187246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:46.198451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:46.203359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:46.204292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:46.205285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 297/610] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:46.212886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:46.324886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:46.344573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b819b-c393-4138-bb64-20acfadde571 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:46.388845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:46.389497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:46.389732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:46.389958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:46.390617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:46.390913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:46.392398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:46.406223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:46.407582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 314/611] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:46.557526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00302ec4-bc18-44f6-9e0b-dfdd16036657 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:46.597003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:46.598041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:46.598521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:46.598988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:46.612774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:46.614505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e2f626db-20e2-4143-8c5b-dd76244828e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:46.635093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:46.636149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:46.637061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:46.637061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:46.637061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:46.643893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 298/612] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:46.651827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:46.823136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602acc7-6140-4bed-982c-5fb0b51e1df8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:46.869334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:46.869757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:46.870043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:46.870314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:46.870758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e6fb1-4853-46ef-bba7-3177c1af66dd None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:46.870963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:46.871063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e6fb1-4853-46ef-bba7-3177c1af66dd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:46.871270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:46.871378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e6fb1-4853-46ef-bba7-3177c1af66dd None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:46.872674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:46.872897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 299/613] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:46.877694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71818ba5-b44c-4594-b6bf-fb83964cbb20 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:46.877971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71818ba5-b44c-4594-b6bf-fb83964cbb20 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:46.878242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71818ba5-b44c-4594-b6bf-fb83964cbb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:46.893548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 315/614] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:46.894335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:46.902713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-71818ba5-b44c-4594-b6bf-fb83964cbb20 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:46.903932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-71818ba5-b44c-4594-b6bf-fb83964cbb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:46.915170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:46.916833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:46.983121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 300/615] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:47.091439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b9b1a-01ff-40ff-95f5-33b930fb06ff None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.109065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e214c-fc56-4c85-93d8-db0c94754a9e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.109399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e214c-fc56-4c85-93d8-db0c94754a9e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.109615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e214c-fc56-4c85-93d8-db0c94754a9e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.110949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 301/616] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:47.116061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.116400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.116602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.138989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:47.139653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.139932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.140199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.148749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:47.150139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:47.162806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:47.163850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:47.164793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 316/617] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:47.169113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:47.169959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:47.176591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.306138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 302/618] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:47.312105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:47.343704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f5f3-892f-4682-819f-fe1dbc6ccfe0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.363240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.399652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:47.400245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.400448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.400713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.401429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:47.401724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:47.403122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:47.442914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 317/619] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:47.446600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.505431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:47.543594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073560de-3644-4b46-9c38-1d49b0595c2e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.551445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af385e1-c1ae-4d1e-94ec-21e86f0d6a87 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.584791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:47.589350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:47.589489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:47.589995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.590274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.590544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.591303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.592167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.592634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.604729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:47.606478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:47.612113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:47.621921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:47.622916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:47.623967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:47.623967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:47.623967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:47.625086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:47.625969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:47.627369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 303/620] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:47.629265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 318/621] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:47.631918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.636360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.779808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57721323-4393-44d3-974d-0b6cf7a518ea None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.787058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171cc84-6e29-43da-b702-847b54269613 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.822364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:47.823435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.823935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.824473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.825073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:47.825554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:47.825657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:47.825932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:47.826197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:47.826539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:47.826855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:47.827155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:47.828552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:47.828658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:47.831417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:47.845451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 304/622] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:47.846607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.849141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 319/623] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:47.850228 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:47.896413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:47.960693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d22a1c-c479-4e15-acd3-7d07ca9f2c98 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.981594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551177d3-b367-4d5e-a082-4d5cba00784f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:47.999797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.000424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.000701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.000964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.013211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.014824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.031262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.032338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.032819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.033305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:48.033698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:48.034615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:48.035536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:48.035536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:48.035536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:48.040256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 320/624] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:48.041411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.054069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:48.061142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:48.062062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:48.062897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:48.065608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 305/625] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:48.071074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.146467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.147891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.159180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d622-889f-4b57-976a-4d48849b0c1f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.194605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93428e8a-e5da-466d-b5ef-68078c5bc8cc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.194794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.195352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.195747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.196022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.203293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.204855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:48.217324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:48.218201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:48.219107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 321/626] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:48.220186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.235855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.236546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.236823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.237107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.237731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:48.238033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.239396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea87902-a2b5-441d-bf22-c07f48d68e72 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.250505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 306/627] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:48.256218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab474ab5-1ed6-43a2-8901-343be9ba8fa5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.256500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab474ab5-1ed6-43a2-8901-343be9ba8fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.256768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab474ab5-1ed6-43a2-8901-343be9ba8fa5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.258790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 307/628] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:48.259500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab474ab5-1ed6-43a2-8901-343be9ba8fa5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.327704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbd51c-bcc6-4187-85f0-898cdb424d59 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.362376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.363203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.363376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab474ab5-1ed6-43a2-8901-343be9ba8fa5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.363798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.364232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.365145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:48.365702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.367631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.380170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 322/629] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:48.380170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.385764 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.396988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.406716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.407320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.407624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.407893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.408579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:48.408884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.410308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714948e9-592c-4753-a21e-92ee93d2a121 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.422778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 308/630] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:48.428454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.428735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.429004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.446941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:48.448405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:48.487812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82043021-5151-4b7b-aa88-c55c43bb29ee None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.531511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 309/631] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:48.531511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.535245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.535976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.536335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.536674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.545617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.547360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:48.559953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:48.561086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:48.562195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 323/632] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:48.563815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.655414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062265e-2ec1-46d7-b63d-ba6c88a43844 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.664694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.667229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.696550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.697183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.697454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.697686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d8f6-e964-4977-911a-ccef29deea0b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.697816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.707864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.709305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:48.723920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:48.724900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:48.725792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:48.725792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:48.725792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:48.728467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 310/633] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:48.729475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.739249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a49d29-fbe7-4bf3-b4c2-7a75df675cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.739871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a49d29-fbe7-4bf3-b4c2-7a75df675cad None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.740142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a49d29-fbe7-4bf3-b4c2-7a75df675cad None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.740409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a49d29-fbe7-4bf3-b4c2-7a75df675cad None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/0587d0fb93ad478fa931141ac9e1a5de/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.757025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a49d29-fbe7-4bf3-b4c2-7a75df675cad None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=0587d0fb93ad478fa931141ac9e1a5de, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.759253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a49d29-fbe7-4bf3-b4c2-7a75df675cad None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.780445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 324/634] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:31:48 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/0587d0fb93ad478fa931141ac9e1a5de/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:31:48.780445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48beef7d-c4d2-4c0a-becc-d96e981ffc5a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.780445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48beef7d-c4d2-4c0a-becc-d96e981ffc5a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.780445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48beef7d-c4d2-4c0a-becc-d96e981ffc5a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.780445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 325/635] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:48.780445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48beef7d-c4d2-4c0a-becc-d96e981ffc5a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:48.857516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12a44d-a3cb-4ea9-8109-3e44f462e51e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.895890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.896498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.896773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.897038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.897780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:48.898608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.900099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c72b-e15f-4b65-9140-b1ff883a4047 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.901517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48beef7d-c4d2-4c0a-becc-d96e981ffc5a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:48.924518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 311/636] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:48.924518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.924518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.924518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.936013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.941102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:48.942295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:48.955756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:48.956482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:48.956763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:48.957035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:48.957691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:48.958000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:48.958583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:48.960175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:48.972756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 326/637] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:48.973545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.022745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 312/638] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:49.022745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.094859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea9629-c4cc-4089-a00c-406be9932b79 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.129948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.130575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.130855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.131125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.145071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:49.152825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:49.153756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:49.154604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:49.155316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 327/639] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:49.155957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.162152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693c69-67dd-4d2c-9a64-2d5425aef68a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.202714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.203348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.203667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.203934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.216416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.223535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.281083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.282472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.292929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07844111-825c-428e-9445-3b9fd045a914 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.331686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.332339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.332623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.332836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.341168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.342600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:49.354924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:49.355871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 328/640] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:49.356814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.394899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 313/641] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:49.396577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.452335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.461225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c553f-e68e-4e5c-b11a-52f31de2341b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.464798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.499662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.500299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.500549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.500822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.501511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:49.501811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.503158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.508106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777fbeb-e1b9-4c5f-982e-a4ff56ad2bf0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.516832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 329/642] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:49.516832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.545823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.546378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.546652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.546924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.557902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:49.562687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:49.563877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:49.564860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 314/643] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:49.569310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.624669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea435303-a94a-4f3f-bc25-3f7d74d76c94 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.669026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.669651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.669925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.670193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.670916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:49.671216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.672673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.690706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 330/644] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:49.691400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d5aa3-474d-4b12-956a-e7feb389f69d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.691544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.704491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.721367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.733050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.733668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.734519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.734519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.744525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.745971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:49.761583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:49.762512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:49.763626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:49.763626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:49.763626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:49.763626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 315/645] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:49.763626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.806504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e22bd-a2ca-4f26-87e2-d11aa7b2d283 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.847019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.847971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.848308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.848648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.858726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.860210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.865637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ccef0-ec0b-4138-ab1c-e4b25e488a2a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:49.874686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:49.875754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:49.876643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:49.876643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:49.876643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:49.876643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 331/646] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:49.876643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.907725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:49.908329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:49.908606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:49.908892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:49.909596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:49.909901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:49.911272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:49.919171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 316/647] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:49.919999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:49.944658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.963606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:49.991861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8c01d-251f-40c5-811b-ffb706590e33 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.032099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.032712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.032990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.033260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.033456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f19780-f33d-47bb-ab1f-f4919f4ac065 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.067931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.068741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.069175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.069581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.070935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:50.071238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.072622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.085255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 317/648] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:50.086672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.110861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.112475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.176890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.192852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.196424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c85b08-d841-4b6a-a5e0-a6ce1f18cf3a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.225193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 332/649] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:50.225193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.235141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.235869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.236144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.236413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.237048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:50.237342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.238697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.250958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 318/650] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:50.251934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.335764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15975b-0d8a-4c98-a187-868cacabc0c0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.363662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517902f5-52e1-40d2-8f7c-5de29e3db9b3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.372557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.373192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.373468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.373739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.384810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.386264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.399933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.400745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.401161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:50.401427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:50.402506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:50.403612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:50.403612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:50.403612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:50.403612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 333/651] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:50.403839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.404208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.414644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.416356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:50.428014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:50.429037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:50.430661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 319/652] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:50.431144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.436608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.442424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.521690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455cc89-57df-4a93-af68-8eb0210470de None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.542404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e15447-34a1-466c-b8b3-aab3b5a1afc0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.556386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.557277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.557692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.558100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.569706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.572618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.581953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.582584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.582856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.583121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.591153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:50.591398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:50.592359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:50.593255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:50.593255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:50.593255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:50.593446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.599608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 334/653] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:50.599608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:50.609753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:50.610702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 320/654] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:50.612202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.680194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.690165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.713848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138852f5-ed3a-4154-8b67-5b0300b2ba19 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.725155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ff704-d7df-4806-98dc-13cd7174c8a7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.749245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.750147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.750602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.751063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.751751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:50.752058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.753469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.762792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.763431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.763743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.764014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.764755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:50.765066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.770813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 335/655] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:50.770973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.771070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.787841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 321/656] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:50.787841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.883151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d291-d131-4822-9982-5fd07f24db89 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.890309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d42ca5-f538-4297-8ed3-7c70547bc0f6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:50.916832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.919358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.920009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.920282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.920666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.921171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:50.921616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.922811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.929640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:50.930241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:50.930519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:50.930787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:50.931541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:50.931667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:50.931843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:50.933378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:50.935516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 336/657] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:50.936936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:50.951742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 322/658] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:50.952566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.040501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c7d5-9f9f-4e92-bb09-877de4d4a80f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.048293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44456cc8-bfd9-43ed-aa9d-6253d2922586 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.072639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.073249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.073521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.073784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.080412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.080887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.080989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.081253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.081517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.082223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=4784d64225054ea9a987581a00b6807b) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:51.082313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.082523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.083968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef2b4-0f1d-40ba-b54b-8e7898b0be9d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.090038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 323/659] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/role_assignments?user.id=4784d64225054ea9a987581a00b6807b&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:51.091229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931677-a9d0-4ce0-b09e-3be8c2c88382 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.091537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931677-a9d0-4ce0-b09e-3be8c2c88382 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.091809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931677-a9d0-4ce0-b09e-3be8c2c88382 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.093191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 324/660] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:51.093717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06931677-a9d0-4ce0-b09e-3be8c2c88382 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:51.096562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:51.097761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:51.098693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 337/661] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:51.099394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.155908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.172558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.219295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e79df2-ad69-4bf6-9557-e97b2792752b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.219720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06931677-a9d0-4ce0-b09e-3be8c2c88382 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.256682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.257818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.258102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.258260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.258659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.259227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.259681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.259952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.266069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.267591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1ae23913-3dbb-4c0c-b881-0c733e91a9fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:51.281527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:51.282520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:51.283397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 325/662] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:51.283935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.284136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.284339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.308218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:51.309543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:51.349741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.351862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.390533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 326/663] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:51.391413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.426056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.438900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.497737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 338/664] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:51.498959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31f3f-1fa7-4409-8b9a-3c37a0b65d36 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.499173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.535423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.536074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.536460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.536667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.537410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:51.537716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.539980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.563818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 327/665] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:51.564699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.626055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9ec2-84d3-4bef-bfe1-b0c934b2329f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.673755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.674565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.675272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.675272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.675941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:51.676299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.677695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.683550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.694270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bd4e5-4b05-4e8d-b991-ce477ed9fd5a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.696404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 339/666] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:51.704536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.731051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.737467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.738301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.738788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.738895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.750227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.751705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:51.767670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:51.768589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:51.769477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:51.769477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:51.769477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:51.769477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 328/667] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:51.770252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.817883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dfac0-1097-4aac-8250-3871de251b32 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.851938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.852824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.852824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.853071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.877347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26f3b7-ba27-4445-a60a-75da5ff786ca None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:51.914406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:51.915165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:51.915441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:51.915738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:51.921689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:51.923124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:51.926287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:51.932280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:51.933226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 329/668] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:51.936487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:51.938449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:51.951424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.042280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 340/669] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:52.043221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.056345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a280312-2e9c-40ee-bbd5-83b01afbc3df None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.090489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.091088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.091364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.091821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.092542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=0d2cf22448884e1f8758b9a448d001ef) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:52.092847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.094277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.100544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 330/670] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:51 2022] GET /identity/v3/role_assignments?user.id=0d2cf22448884e1f8758b9a448d001ef&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:52.102080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.160127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd3f1f-d89a-45a6-a98a-1492ebbc6ff4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.187666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.197921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.204719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.205338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.205618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.205889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.206543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:52.206843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.208337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.221449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 341/671] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:52.222327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.248711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87310a-16e7-4590-9d04-5b7a404fc253 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.292020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.292974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.293433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.293931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.294842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:52.295337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.297101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.307185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 331/672] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:52.308358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.368685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af85aa-76f4-4b24-9c73-ebbe62d0d82b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.423450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.424112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.424385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.424652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.435951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.437479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.443892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e025e-5ac3-43aa-b137-12b2ff3436ac None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:52.457622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:52.458700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:52.459621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:52.459621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:52.459621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:52.460218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 342/673] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:52.461772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.475277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.483409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.484220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.484494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.485051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.493404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.495698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.498028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:52.510558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:52.511467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:52.512415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 332/674] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:52.519502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.587264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3c61-748a-40a3-a87a-2cd15a68153c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.624740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.625365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.625640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.625905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.635573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9c634-a792-4f35-9ea9-cc8ec41063a9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.637386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.638945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:52.655306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:52.657529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:52.659550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:52.659550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:52.659550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:52.659550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 343/675] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:52.659550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.679665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.680470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.680896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.681337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.682231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:52.682711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.684337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.697633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 333/676] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:52.698658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.751726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.754519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.774703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540b2b2-9ef4-4bdc-9b41-a71d99f2c15a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.806554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c5ee9-fd06-478b-aa3c-0c651da04b27 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.811052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.811765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.812108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.812441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.813196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:52.813586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.815066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.833482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 344/677] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:52.834639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.846208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.847122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.847680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.848122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.848946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:52.849555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.851189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.868072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 334/678] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:52.869212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:52.936823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc5b1f-c9a8-47ff-9bf2-3d4204a7fc12 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.972648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:52.973286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:52.973672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:52.974026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:52.981142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:52.982682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:52.987427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdded7a-9da1-4842-bd1e-e5e411b99311 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:52.996681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:52.997605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:52.998513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 345/679] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:52.999003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:52.999989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:53.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:53.030921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:53.031701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.031927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.032203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.032933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=24bf5028fe8d4a619897cf34ba3ff809) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:53.033242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:53.034673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:53.047511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 335/680] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/role_assignments?user.id=24bf5028fe8d4a619897cf34ba3ff809&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:53.047511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:53.213339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fc75-e623-4d1e-a11e-be4a379fc7a2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:53.230472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745d846-22d0-4601-838f-10ec55264fb8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:53.286141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:53.287151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.287660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.288144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.296842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:53.296842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.296842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.296842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.297541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:53.297845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:53.298904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:53.299292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:53.300844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:53.313374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:53.314361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:53.315311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:53.315892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 336/681] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:53.317875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 346/682] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:53.319537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:53.325946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:53.357939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:53.397531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:53.442669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba25d-17fc-4848-9a15-e0bd761d3210 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:53.443951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69140-25cd-40b0-8529-698dd7d2ba34 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:53.482396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:53.483587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.484473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.485163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.486077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=7727b52cd17b41708b243dac7b4f2c16) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:53.486642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:53.488372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02cd3f-aa85-4986-98ca-d781f888164b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:53.488984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:53.489655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.490001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.490332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.491401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:53.492330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:53.494357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:53.498460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 337/683] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:53 2022] GET /identity/v3/role_assignments?user.id=7727b52cd17b41708b243dac7b4f2c16&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:31:53.499980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab689-77a7-4a55-b31e-09f57312a315 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.500445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab689-77a7-4a55-b31e-09f57312a315 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.500914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab689-77a7-4a55-b31e-09f57312a315 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.507519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 338/684] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:53.511299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.511795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.512273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.519132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 347/685] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 00:31:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:53.528621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:53.539437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:53.541326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:53.649124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e732e0-84a1-4f81-b312-2161f6f10432 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:53.680389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:53.701636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:53.702253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:53.702528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:53.702797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:53.723524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 339/686] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:53.786231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:53.787626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:53.887906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 348/687] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:53.893325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:53.924718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.010840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22044670-a5af-4af4-9d93-3e9af7967fac None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.046750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.062286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.063020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.063365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.063719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.083523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.083828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.086485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:54.119442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:54.120624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:54.121702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:54.121702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:54.121702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:54.121702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 340/688] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:54.127004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.225132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.229178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36abce-0d78-4bb3-a549-1931ff7d3c52 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.234855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02c928-f87b-4779-b553-ea447a696355 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.295236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.295880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.296160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.296431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.297242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:54.297495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.298977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.304205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.305087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.305559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.306020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.312209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 341/689] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:54.317757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.327433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:54.332641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:54.333582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:54.334415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:54.339892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 349/690] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:54.341082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.461248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554c6c7-9f27-46a2-9ee0-24efafef1b61 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.497009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.499434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.500090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.500419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.500662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.508753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.510213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.512951 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:54.529275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:54.530216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:54.531188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 342/691] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:54.536432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.537189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c615c-98f8-4724-9eb3-89b0ec59d854 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.576192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.576810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.577089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.577364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.578037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:54.578342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.579731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08af05-b153-4c21-bd64-5770c6b1f72a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.590392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 350/692] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:54.591763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53998a9-87cc-4ce8-b7b8-722e99811db7 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.592052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53998a9-87cc-4ce8-b7b8-722e99811db7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.592327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53998a9-87cc-4ce8-b7b8-722e99811db7 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.593529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 351/693] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:54.595899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53998a9-87cc-4ce8-b7b8-722e99811db7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.691845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9228-51c1-4aeb-8e12-17be41282fba None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.722558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53998a9-87cc-4ce8-b7b8-722e99811db7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.729724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.730512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.731017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.731431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.732347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:54.732903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.734601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51620-14f8-4f4a-947a-da11391c42d6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.753020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 343/694] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:54.754826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.755328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.755772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.760465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.761111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.761393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.761663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.773351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.774859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.777062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:54.777719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.779808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:54.791262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:54.792485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:54.793519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:54.793519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:54.793519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:54.797113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 352/695] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:54.797960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.845882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:54.867918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 344/696] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:54.869307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:54.910456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1a254-0a65-4059-8476-af8cc4dee55d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:54.945713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:54.946324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:54.946612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:54.946889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:54.947643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=0587d0fb93ad478fa931141ac9e1a5de) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:54.947956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:54.951016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:54.961905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 353/697] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/role_assignments?user.id=0587d0fb93ad478fa931141ac9e1a5de&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:55.030256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea927edd-e2ed-42c1-8e94-3cf1572fd78c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.094276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.094914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.095191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.095462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.096368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:55.096616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.098036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.114965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 345/698] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:55.120517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:55.210906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:55.235441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:55.289817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c7c34-a0fd-4901-b4f9-89db20e44654 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.319531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:55.337726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.338364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.338640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.338908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.348853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.350282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:55.363402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:55.364657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:55.365714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 354/699] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:31:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:55.370577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:55.396846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc012-2b4b-4042-b557-153621b0766a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.448555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.449679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.450276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.450728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.465926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.468245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:55.490147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:55.491129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:55.492139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:55.492139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:55.492139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:55.493017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 346/700] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:55.500129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:55.504397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d03b1d-f905-42d3-bb41-859128c264bd None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.530191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:55.547650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.548376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.548720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.548989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.550271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:55.550575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.552483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.575091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 355/701] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:31:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:55.581402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:55.622465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a6f3c-1f12-4db6-a7a7-83ba9e7404c3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.624101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:55.676631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.677874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.678374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.678859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.679803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:55.680603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.682225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.698153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 347/702] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:55.709273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:55.765812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636c786-3b59-45ea-9688-af1542e19327 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.844245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.844860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.845145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.845419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.862040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541dc61-0af2-4639-8e33-4a792dc90a96 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:55.892869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:55.902835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb1670-b979-4334-98e4-91587a54bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:55.903734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1670-b979-4334-98e4-91587a54bbd8 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.904159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1670-b979-4334-98e4-91587a54bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.904634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1670-b979-4334-98e4-91587a54bbd8 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.905424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb1670-b979-4334-98e4-91587a54bbd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.907026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb1670-b979-4334-98e4-91587a54bbd8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.915026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:55.916503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:55.930901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 348/703] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:31:55 2022] POST /identity/v3/users => generated 254 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:31:55.941282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a2cff-90bd-400a-b75f-7988d7355adb None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.941631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a2cff-90bd-400a-b75f-7988d7355adb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.941954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a2cff-90bd-400a-b75f-7988d7355adb None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.943788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 349/704] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:55.950077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:55.950399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:55.950613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:55.990754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:55.992384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:56.024694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:56.032188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 356/705] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:31:56.038691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.138083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 350/706] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:56.144835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:56.164837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a192290-fd95-4ed0-b5af-0462beaa094e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:56.206440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:56.207090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.207368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.207670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.220034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:56.221515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-f15ec159-5348-47a5-bec0-d6250c816442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:56.238753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:56.239842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:56.240742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:56.240742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:56.240742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:56.243521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 357/707] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:56.248909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.345051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131d694-21eb-4dd4-8963-5643b11d8fbe None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.345378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131d694-21eb-4dd4-8963-5643b11d8fbe None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.345587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131d694-21eb-4dd4-8963-5643b11d8fbe None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.347205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 358/708] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:31:56.352410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.352658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.352932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.372875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:56.374060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:56.374822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394aa55-f263-41a6-8bfb-446879884324 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:56.436288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:56.437256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.437741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.438231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.439086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:56.439650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:56.441244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:56.454552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 351/709] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:56.457507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.458395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 359/710] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 00:31:56.460465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.481849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:56.597398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:56.641595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8478-9c8f-4991-8056-ea8cec6f904a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:56.660969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560db1c4-4097-49fe-81d3-a312cd95cf05 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:56.705666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:56.706225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.706502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.706771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.719530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:56.720983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:56.727424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:56.728093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.728371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.728645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] PATH_INFO: `/v3/services/network` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.729231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:56.730593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:56.751610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:56.752669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:31:56.753515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:31:56.754269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:56.755056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:31:56.756009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:56.759597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 360/711] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:56.760459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.763434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 352/712] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/services/network => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:56.765623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.859984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:56.870755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89e9f4-1eb4-43ad-97ca-0af177a6c22d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:56.891940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:56.900070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111212e-54c4-4b79-9b96-f03b779e0818 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:56.908660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:56.909263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.909538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.909812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.910585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:56.910831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:56.912213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:56.917636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 361/713] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:56.918507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:56.944427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:56.946078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:56.946568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:56.947295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:56.948887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:56.949404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:56.951998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:56.968759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 353/714] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:56.969974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:57.028280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00479d24-72dc-48d2-90af-4f5d45dfcec5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:57.065106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:57.065796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.066070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.066339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.066967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:57.068356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:57.080482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:57.081492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:57.082384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 362/715] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:57.083185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:57.113452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e44b-fe12-4528-a210-8a52dbdaa24b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:57.136352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:57.156593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:57.165352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:57.166800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.167293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.167802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.169090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] RBAC: Adding query filter params (type=network) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:57.170057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:57.171999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:57.181024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 354/716] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:31:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:57.182380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:57.192978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488dfbb-1536-4ee6-92d1-168b30dbd37c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:57.252160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:57.252694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.252968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.253232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.253974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=4784d64225054ea9a987581a00b6807b) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:57.254275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:57.255779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:57.268644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 363/717] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:31:57 2022] GET /identity/v3/role_assignments?user.id=4784d64225054ea9a987581a00b6807b&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:57.332443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9952ba-8b58-4804-ba82-d5b60fbf529f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:57.376676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:57.377295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.377561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.377828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.378467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:57.378763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:57.380273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c214445-8188-4f55-81fd-8f94e0d0102f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:57.386035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 355/718] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 00:31:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:57.392660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:57.530500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899d360-ee32-42f1-8464-20c8b0a104b1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:57.544487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:57.567469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:57.568150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.568426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.568733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.569355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:57.570177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:57.571594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530fc3a-0b97-4379-bb92-73737a8b93f8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:57.577470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 364/719] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 00:31:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:57.849455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adfa4e-93d8-461d-8ba7-c2a130809ad9 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.849841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adfa4e-93d8-461d-8ba7-c2a130809ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.850039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adfa4e-93d8-461d-8ba7-c2a130809ad9 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.851590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 356/720] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:57.857236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:57.857785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:57.858266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:57.878064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:57.880065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:57.961442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 365/721] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:58.260239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adfa4e-93d8-461d-8ba7-c2a130809ad9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:58.342920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:58.409282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adfa4e-93d8-461d-8ba7-c2a130809ad9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:58.459201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:58.459879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:58.460684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:58.460964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:58.477057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:58.482355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:58.483277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:31:58.484147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:58.487844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 357/722] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:58.492576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:58.652354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dcfb2-c1ff-4a58-b350-0e756aafb81c None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:58.666427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:58.712852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:58.713470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:58.713747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:58.714014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:58.714679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:58.714977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:58.716356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:58.731686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 366/723] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:58.737020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:58.885900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fac680-5e33-456a-a59b-947d9fc16b41 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:58.912354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:58.921675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:58.922582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:58.923235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:58.923740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:58.934739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:58.936524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-dc3c61bb-2131-42ea-9510-b93f210971fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:31:58.951711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:58.952774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:31:58.953710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:31:58.953710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:31:58.953710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:58.955893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 358/724] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:58.961648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:59.030297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4a72e-f981-404b-8023-9b9de454932c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.030943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4a72e-f981-404b-8023-9b9de454932c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.031429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4a72e-f981-404b-8023-9b9de454932c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.033238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 359/725] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:31:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:31:59.041103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.041385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.041660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.071550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:31:59.072751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:31:59.104236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb74271-0d36-4852-aeb2-86bf78321c70 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:59.162530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:59.163054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.163362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.163664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.164302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:59.164604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:59.165954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:59.192318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 367/726] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:59.197025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:59.204259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 360/727] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:31:59.231899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:59.304909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23e309-ab0b-4d4e-a602-dd133ce043c2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:59.340130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:59.340937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.341222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.341495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.348944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:59.350380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:31:59.365863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:59.367402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:31:59.368960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 368/728] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:31:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:31:59.376614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:59.485955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2b18b-4b3a-4d9e-bf0a-8c23ac1e84b4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:59.487553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:59.531908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:59.532517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.532795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.533119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.533859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:31:59.534223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:31:59.535759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:31:59.553440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 361/729] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 00:31:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:31:59.560749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:59.605846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:59.730483 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db88e2-a3df-4d1e-b445-d9a88c42acbc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:59.742116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:59.771023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:59.772465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.773536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.774044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.783021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4590a2-4e86-4eb9-a056-67413c665b13 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:31:59.785579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:31:59.791852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:31:59.792778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:31:59.793637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:31:59.802767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 362/730] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:31:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:31:59.804787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:31:59.848077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:31:59.857526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f01ba7-5746-42ea-be48-be0b83eb79f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:31:59.859579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f01ba7-5746-42ea-be48-be0b83eb79f1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:31:59.860116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f01ba7-5746-42ea-be48-be0b83eb79f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:31:59.861554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f01ba7-5746-42ea-be48-be0b83eb79f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:31:59.967703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbc034-6b41-45d0-b578-b981669b67df None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.010629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.011638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.012131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.012632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.013899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:00.014415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.016037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.031189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 363/731] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:31:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:00.032781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.081134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f01ba7-5746-42ea-be48-be0b83eb79f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.082585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f01ba7-5746-42ea-be48-be0b83eb79f1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.142813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.163076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbb4fc-199e-404c-86a0-4f92ad8cecdf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.176302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.189364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 369/732] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 00:32:00.190621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4f86c-ce1c-4212-b1cb-471ed5bb0a6a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.190909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4f86c-ce1c-4212-b1cb-471ed5bb0a6a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.191180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4f86c-ce1c-4212-b1cb-471ed5bb0a6a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.195697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 370/733] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:00.195697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb18a4d-e775-4e73-8c25-599f7c545fb7 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.195697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb18a4d-e775-4e73-8c25-599f7c545fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.195697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb18a4d-e775-4e73-8c25-599f7c545fb7 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.199457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 371/734] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:00.200259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb18a4d-e775-4e73-8c25-599f7c545fb7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.206623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.207262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.207571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.207846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.219897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.221373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-608c56cc-7c3c-49af-8c67-b3600d189257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:00.235994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:00.236966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:00.237941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:00.237941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:00.237941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:00.237941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 364/735] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:00.239323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e7391-fe99-44ab-8719-f3d38bea329a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.239323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e7391-fe99-44ab-8719-f3d38bea329a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.239323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e7391-fe99-44ab-8719-f3d38bea329a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.255828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-767e7391-fe99-44ab-8719-f3d38bea329a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:00.257518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-767e7391-fe99-44ab-8719-f3d38bea329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:00.302531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb18a4d-e775-4e73-8c25-599f7c545fb7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.323539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 365/736] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:32:00.324923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.325208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.325476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.342619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:00.343818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:00.348589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.349221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.349502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.349774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.361438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.362873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-a755165a-94a3-44d3-9dd8-569bb88db0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:00.378488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:00.379422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:00.381030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:00.381030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:00.381030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:00.381030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 372/737] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:00.381287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa9337-a975-4041-a0d5-cae1c775bcb8 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.381354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa9337-a975-4041-a0d5-cae1c775bcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.382411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa9337-a975-4041-a0d5-cae1c775bcb8 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.387979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 373/738] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:00.387979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa9337-a975-4041-a0d5-cae1c775bcb8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.405890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.414888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 366/739] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:32:00.415913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.451832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.524220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa9337-a975-4041-a0d5-cae1c775bcb8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.536334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118fea7-74f2-467a-84d0-08e2a36bb6c0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.569858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.570467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.570820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.571146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.571870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:00.572231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.573656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a19959-ba3e-4f85-bb35-1b74051a6c7d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.574255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.575051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.575542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.576004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.576899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:00.577448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.579064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.588054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 374/740] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:00.589808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.590092 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.590364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.593802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 367/741] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:00.594964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.611447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:00.613048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:00.661066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.686763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 375/742] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 00:32:00.687778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.715248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b76f2-9d9a-4422-a45a-fa2848a38eff None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.724759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.749627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.750255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.750535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.750819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.759152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.760634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:00.771793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:00.772810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 368/743] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:00.773769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.803070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10f056-7d56-46fb-80f3-a7c3ee4f8a33 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.838235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.838869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.839149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.839420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.846036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.847465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:00.858882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:00.859822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:00.860717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 376/744] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:00.861792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.885534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009397ad-94b1-4857-8cb6-9a637da77518 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:00.902864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.927560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:00.928223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:00.928506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:00.928812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:00.929460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:00.930845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:00.947578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 369/745] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:00 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:00.956216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:00.967637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:00.980960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019fbd9-069a-4d5e-89dd-0c2dd9a907da None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.041045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.041710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.041995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.042271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.053828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:01.058938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:01.060032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:01.060865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 377/746] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:01.061569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.076834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d870e5-81a6-4433-90ee-043009c181bd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.114359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.115234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.115522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.115803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.116577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:01.116869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.118368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.139576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 370/747] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 00:32:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:01.139754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.174885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:01.181706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898eb26-9ad4-4d23-a435-b2857e05bf2b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.218332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.218968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.219242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.219572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.220242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:01.220544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.221966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.233680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 378/748] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:01.234715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.245701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:01.254798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d55c4-038c-46a8-badd-c34502f498c7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.291357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.292148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.292528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.292886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.293645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.295110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.309534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 371/749] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:01 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:01.316576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.355436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb735b3-71b3-4fab-a8b2-96074740bd9f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.419135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.420309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.421201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.421756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.422725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:01.423183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.425140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.439861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 379/750] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:01.441081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.463469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:01.482727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32462a-27c7-4a70-a7ea-f135a71d581a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.523455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:01.532211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.532833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.533122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.533395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.574363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14026f-5b1e-47aa-8167-a6fbaa1fa02c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.610453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.611052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.611325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.611444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.611661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.612338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=0587d0fb93ad478fa931141ac9e1a5de) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:01.612641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.612894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.614031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.620781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 380/751] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/role_assignments?user.id=0587d0fb93ad478fa931141ac9e1a5de&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:01.621706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.716000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:01.723058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 372/752] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:01.729705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.813557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb680e3-dfbc-4374-bdf2-aef41b72df72 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.840180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af88c-fe77-47ea-b7d2-b0aa553fcd94 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:01.857171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:01.878823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.879172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:01.879496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.879786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.879886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:01.880055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:01.880147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.880334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:01.893504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.894689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:01.895288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.896178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:01.916782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:01.917805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:01.919336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:01.919336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:01.919336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:01.919336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 381/753] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:01.924669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:01.932841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:01.934036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:01.935323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:01.935323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:01.935323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:01.935323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 373/754] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:01.939679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.026061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:02.111035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce6030-ed8d-4412-8523-4eae6c0d034d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.142333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248c0e8-c01e-4a46-a423-8a6c079a691b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.172785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.173401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.173683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.179689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.180362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:02.180672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.182058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.198425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.199162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.199447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.200108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.201291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:02.201854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.203447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 382/755] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:02.204490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.208691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.219095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 374/756] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:02.224762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.289896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:02.346879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cf880-f1bc-4a5a-9cb5-f325c7d85334 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.349624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:02.392846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.393502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.393761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.394043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.395284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81156ae-8e26-4db5-879a-38293ee8939f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.412578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.416157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.432107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.432654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.432925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.433191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:02.438274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:02.440340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:02.442552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:02.447503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 375/757] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:02.447503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.458121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.459717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:02.483388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:02.485754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:02.488051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 383/758] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:32:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:02.492602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.572459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d57aba-ba3d-4561-bd3c-87560a3c38e0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.612908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.613571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.613856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.614128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.614830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:02.615136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.617828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.633896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 376/759] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:02.640008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.675346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:02.685921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:02.694864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dc761-71c4-4183-9d0f-3b57637ef525 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.765163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.765780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.766057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.766330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.767073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:02.767379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.768838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.769564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923776ae-fba0-4b59-92d0-387d1a058eb1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.804420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 384/760] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:32:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:02.805375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:02.813160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.813775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.814258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.814540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:02.815278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=5287027e82ff49e78cae60565bdcc611) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:02.815621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:02.817351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:02.823983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 377/761] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:02 2022] GET /identity/v3/role_assignments?user.id=5287027e82ff49e78cae60565bdcc611&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:02.949998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f8e2-d703-402f-ba6f-7494c489ed2b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:02.997510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:02.998490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:02.998945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:02.999409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.011514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:03.012996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:03.039387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:03.042049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:03.044230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:03.044230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:03.044230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:03.044230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 385/762] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:03.051214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:03.195866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:03.218044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f5256-2a7d-4d3e-b642-ac31ac5eed21 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:03.270594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:03.271208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.271493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.271831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.272550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:03.273341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:03.274715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:03.289683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 378/763] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:03.295333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:03.315025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:03.427878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae0178-369e-41e7-93e7-23169b611984 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:03.474219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:03.474773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.475050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.475315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.485759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:03.487199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:03.509141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:03.511335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:03.513499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 386/764] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:03.520361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:03.641777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7898e-956a-4802-892b-aa2a3329e647 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:03.688571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:03.689211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.689490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.689759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.690488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:03.690794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:03.692346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06817-2f82-4cf9-8cff-8d0ea78e400c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:03.710745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 379/765] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:03.712573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:03.855789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:03.902520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1572434-3c97-4873-a74f-efd4f2163fa5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:03.947207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847e2fa-e681-482e-a7d9-e8a9c1650a7e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.947549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847e2fa-e681-482e-a7d9-e8a9c1650a7e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.947998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847e2fa-e681-482e-a7d9-e8a9c1650a7e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.949298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24231f8e-d558-4d83-b2ff-5f7759893d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:03.949950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24231f8e-d558-4d83-b2ff-5f7759893d10 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.950225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24231f8e-d558-4d83-b2ff-5f7759893d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.950492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24231f8e-d558-4d83-b2ff-5f7759893d10 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/4784d64225054ea9a987581a00b6807b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.950742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 380/766] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:03.951823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca35f-ed91-4062-9048-148eeff39165 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.952107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca35f-ed91-4062-9048-148eeff39165 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.952381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca35f-ed91-4062-9048-148eeff39165 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.952490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24231f8e-d558-4d83-b2ff-5f7759893d10 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=4784d64225054ea9a987581a00b6807b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:03.953804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24231f8e-d558-4d83-b2ff-5f7759893d10 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:03.956516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 381/767] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:03.958169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659f465-6867-4882-af1c-3e12248239ab None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.958462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659f465-6867-4882-af1c-3e12248239ab None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.958884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659f465-6867-4882-af1c-3e12248239ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.969135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 387/768] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:03 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/4784d64225054ea9a987581a00b6807b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:32:03.970612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616174e-86fa-4984-9b52-c761148c35d7 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:03.971095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616174e-86fa-4984-9b52-c761148c35d7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:03.971628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616174e-86fa-4984-9b52-c761148c35d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:03.989220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4659f465-6867-4882-af1c-3e12248239ab None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:03.990434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4659f465-6867-4882-af1c-3e12248239ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:03.992174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7616174e-86fa-4984-9b52-c761148c35d7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:03.993590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7616174e-86fa-4984-9b52-c761148c35d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:03.995833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:04.091212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 388/769] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:04.116843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 382/770] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:04.293680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265bf66-ffd3-461d-b64b-49d373980cee None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:04.294116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265bf66-ffd3-461d-b64b-49d373980cee None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:04.294482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265bf66-ffd3-461d-b64b-49d373980cee None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:04.296100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 389/771] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:04.301355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:04.301639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:04.301912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:04.324584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:04.326001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:04.413246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 383/772] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:04.436724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265bf66-ffd3-461d-b64b-49d373980cee None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:04.463787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:04.539986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:04.544670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:04.565949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f407a-3c3c-444a-abd3-eee748a90e02 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:04.601336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:04.601967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:04.602246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:04.602520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:04.603106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:04.604511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:04.615076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:04.616655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:32:04.617530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:04.622950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 384/773] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 15 00:32:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:04.625661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:04.627200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265bf66-ffd3-461d-b64b-49d373980cee None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:04.671041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:04.672306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:04.673559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:04.674183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] PATH_INFO: `/v3/services/network` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:04.676065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:04.683504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:04.692392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:04.693395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:32:04.694318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:04.698484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 390/774] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:04 2022] GET /identity/v3/services/network => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:04.699597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:04.817223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d23f79-7478-4730-92b9-6794e2fcf7db None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:04.837083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed2d73-478e-4d4e-8f09-5a03dad279ae None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:04.849832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:04.852927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:04.853650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:04.854058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:04.854440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:04.855158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:04.855583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:04.857138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:04.866537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 385/775] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 00:32:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:04.866537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:04.871222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:04.886933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:04.887563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:04.888085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:04.888355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:04.889376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] RBAC: Adding query filter params (name=network) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:04.889671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:04.891029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:04.898630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 391/776] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:04.899772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:04.995774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b94c9-f92a-4ad8-8443-f32e80e154fd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.030954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76cb5-eff6-4168-be03-c4e214f7f10e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.032979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.034527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.034907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.035278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.048402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:05.055388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:05.056411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:05.057375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:05.059588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 386/777] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:05.060872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.068342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.068990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.069883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.070153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.070783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:05.071081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.072800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.080405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 392/778] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 00:32:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.081428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.128003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:05.140345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:05.178211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f8620-b886-4248-88d7-a55d25babae3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.191319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bfd78-3e89-49c6-a4cf-f3502fcd5da0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.215316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.216258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.216681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.217136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.217921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] RBAC: Adding query filter params (type=network) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:05.218358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.220145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.225400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 387/779] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.226232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.229143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.230087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.230714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.231189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.232085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:05.232608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.234175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.244421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 393/780] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.245680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.338308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ddba3-4cf5-4e03-8964-249ec3309e54 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.357947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2905c-ce26-4bc4-a0e9-122251378630 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.359169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:05.375283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.376235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:05.377095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.377606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.378242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.379188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] RBAC: Adding query filter params (interface=public, service_id=28a0a0a928a04687bcc4b4b8ecd1e22c, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:05.379743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.381254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.392234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 388/781] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/endpoints?service_id=28a0a0a928a04687bcc4b4b8ecd1e22c&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.393315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.405845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.406460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.406735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.407003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.407735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] RBAC: Adding query filter params (interface=public, service_id=833c4ef7ff5941aab17d6e8c0d69f34a, region_id=RegionOne) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:05.408038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.409390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eb627-43ff-4d4d-a71f-7dcff6f05c04 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.416965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 394/782] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/endpoints?service_id=833c4ef7ff5941aab17d6e8c0d69f34a&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.418127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67aabb-dd5a-458b-9df1-7022b1254aff None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.418403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67aabb-dd5a-458b-9df1-7022b1254aff None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.418674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67aabb-dd5a-458b-9df1-7022b1254aff None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.423520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 395/783] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:05.423520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67aabb-dd5a-458b-9df1-7022b1254aff None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.512815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be712827-00bb-4399-acd5-08d5f5a1e3bb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.528182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67aabb-dd5a-458b-9df1-7022b1254aff None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.548837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.549781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.550334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.550608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.560976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.561575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.561850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.562114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.562471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.562709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.563964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.564064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.576918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 396/784] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/services => generated 772 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.576918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-9cb75b3f-e7bd-41f0-a667-20458fdccb7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:05.590154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:05.591123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:05.592052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:05.592052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:05.592052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:05.592222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 389/785] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:05.593473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.593756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.594029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.620556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:05.622898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:05.630413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:05.669109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:05.718406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 390/786] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:05.719519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aec047-2116-4bd5-9356-4124c5c749a0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.719940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:05.769885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.770504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.770800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.771080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.772195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.774263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.781099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 397/787] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.846856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b38b8-1fa3-4dba-9641-f51bdfd2d461 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:05.887829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:05.892291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:05.892788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:05.893203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:05.894013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:05.894463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:05.896046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:05.916545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 391/788] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:05.924616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.050083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.065293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128673-59d6-4830-a127-d795a311c252 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.073123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:06.083701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:06.112073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.112889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.113175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.113449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.123770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:06.125909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:06.140878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:06.141848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:32:06.142791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:06.145285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 398/789] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:06.150911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.194885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9408f-028f-43ff-a266-d2c20c9e9071 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.257034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.257893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.258167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.259759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.279312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:06.287665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:06.289305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:06.290598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 392/790] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:06.295821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.329381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729482-da2c-4637-9a1c-6e6d71d967f9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.378360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.381493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.382111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.382610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.385386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:06.385971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:06.389555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317bbaf-97fb-44f8-9661-2ac66d80b08b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:06.451470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:06.468791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20940d42-bfa1-4dbd-a0c1-294c5d679568 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.520881 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.521952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.522514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.522992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.523933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:06.524480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:06.526178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ce80f-e6fe-49b2-9c99-e16b7a140dc0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:06.536403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 393/791] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:06.538580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790cb83-63c9-4756-a579-a1485587a8c5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.539138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790cb83-63c9-4756-a579-a1485587a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.539662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790cb83-63c9-4756-a579-a1485587a8c5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.545378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 394/792] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:06.546625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790cb83-63c9-4756-a579-a1485587a8c5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.569984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 399/793] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:06.571309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.571619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.571888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.594956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:06.596626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:06.615084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:06.670287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790cb83-63c9-4756-a579-a1485587a8c5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.687106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 400/794] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:06.687106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.743465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.744960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.745825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.746742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.778495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:06.780456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:06.794026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b928b8-de65-4b76-b761-3f96b9dade04 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.796524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:06.809612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:06.810624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:06.811541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:06.811541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:06.811541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:06.811541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 395/795] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:06.817780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.841771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.842777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.844443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.845118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.869384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:06.870840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:06.902259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:06.903245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:06.904183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:06.904183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:06.904183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:06.911905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 401/796] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:06.917833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:06.939913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:06.954133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd303ab8-9061-43d7-834e-c25843c2f765 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:06.996688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:06.997302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:06.997573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:06.997839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:06.998475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:06.998771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.004073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.030361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 396/797] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:07.035954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.115556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.120090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccbfa2-1e20-4337-ad83-02b639e92e3b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.158343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968679d-0c26-4658-b46f-ce233725712c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.159966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.160947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.161428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.161896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.162731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:07.163313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.164889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.177880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 402/798] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:07.179246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.192567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.193115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.193390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.193657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.201614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.203055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:07.215055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:07.216052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 397/799] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:07.217588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.258235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.298975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13452de1-cc0a-4c5e-9271-6bb45ccc2cbc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.319827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ea897-6290-4d28-a3d3-0c9c97c92c64 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.337550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.338541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.339068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.339609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.340677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.356672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:07.358423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.359046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.359319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.359618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:07.364784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:07.365717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:07.366642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:07.367281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.368552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.374912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 403/800] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:07.376236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:07.380607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:07.382688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:07.385829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:07.387729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 398/801] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:07.387873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.492150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952602a-7cc7-46c4-b1a2-be9bdbe399a3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.503302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372b297-9bfb-466b-9448-afc564d46907 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.526807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.527245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.527547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.527821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.528479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:07.528776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.530134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.534123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.544112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 399/802] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:07.544112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.550843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.551744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.552279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.552769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.553906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:07.554433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.556214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.575989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 404/803] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:07.577314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.585481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.643667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85722e-a078-4fce-b326-13ba8c5a39b1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.679358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.680193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.680468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.680731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.681419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:07.681871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.683345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.698446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c299f79-5303-4c42-ac04-26006bcb56b1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.701870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 400/804] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:07.702987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.742074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.743130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.743662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.744160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.755981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.758245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:07.776173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:07.777164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:07.778078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:07.778078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:07.778078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:07.780047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 405/805] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:07.786884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:07.808764 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8409945-e8b0-4128-99bf-67e9426a008e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.810281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.829955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:07.846433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.847043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.847328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.847626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.857873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.859327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.909927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb6a6e-bc3c-4ddd-a33c-f6daeb50c022 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:07.953821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:07.954908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:07.955411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:07.955950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/0587d0fb93ad478fa931141ac9e1a5de/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:07.958368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=0587d0fb93ad478fa931141ac9e1a5de, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:07.960763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:07.979190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 406/806] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:07 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/0587d0fb93ad478fa931141ac9e1a5de/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:32:07.980642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:08.010652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:08.012875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:08.014712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:08.014712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:08.014712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:08.014712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 401/807] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:08.017461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.109617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.120759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ca8d9-6704-41b8-84fb-2eee997682fc None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.168437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.169858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.170389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.170900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.171848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:08.173067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.175169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27145eeb-35da-4ba7-bd2a-560e42f0491d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.191531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802ce95-694c-4b0d-bb62-4f49fa46e55d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.196537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 407/808] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:08.198158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b5d6b-780f-4c50-989a-83c6e7016fda None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.198551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b5d6b-780f-4c50-989a-83c6e7016fda None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.198927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b5d6b-780f-4c50-989a-83c6e7016fda None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.200750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 408/809] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:08.205904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b5d6b-780f-4c50-989a-83c6e7016fda None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.231502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.232134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.232406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.232669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.233306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:08.233605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.234958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fc25f-501b-4d3f-9e7c-3fe5032d61ac None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.248310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 402/810] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:08.249311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.249590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.249857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.271538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:08.272854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:08.287154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.329287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b5d6b-780f-4c50-989a-83c6e7016fda None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.359520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 403/811] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:08.359520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.369813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.370605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.370949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.371217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.378981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.380560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ef93fc61-686b-4a82-b1c2-f540ab237309 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:08.393863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:08.394848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:08.396310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:08.399770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 409/812] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:08.401028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687606d-3097-4b4d-8173-099519f6c572 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.401309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687606d-3097-4b4d-8173-099519f6c572 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.401578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687606d-3097-4b4d-8173-099519f6c572 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.406834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 410/813] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:08.406834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687606d-3097-4b4d-8173-099519f6c572 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.426233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.479027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c2e3c-219b-4d78-90be-55fdd698be9a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.517529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.518607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.519116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.519767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.524386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687606d-3097-4b4d-8173-099519f6c572 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.528108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.529839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-c6b44590-28ed-40c0-a54a-759eda535252 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:08.546648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:08.547822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:08.550305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:08.555502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 404/814] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:08.555502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.555502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.555502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.562972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.564668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.565461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.565877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.566294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.567162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:08.567636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.570210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.575870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:08.577048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:08.589942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 411/815] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:08.590966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.671604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 405/816] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:08.672668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.676742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.704096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50158ad3-7c74-4761-9e20-f774810060fb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.740525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.742087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.742536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.742975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.743863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:08.744356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.745937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.758907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 412/817] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:08.759989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:08.774079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6257-bbb6-4edc-a978-48bb7b7b2d8c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.820049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.820570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.820840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.821102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.843842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.867061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0af7d4-1ad9-4ea2-99c8-4da815ed5674 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:08.905821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:08.906443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:08.906721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:08.906997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:08.907580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.909023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.910890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:08.917012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:08.918756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:08.919684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:08.920888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 00:32:08.921955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:08.925037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 413/818] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:08.926014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.034357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61259903-dcee-4f46-a31a-45e32c463290 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.035658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 406/819] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:09.038631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.074830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.075509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.075776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.076041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/5287027e82ff49e78cae60565bdcc611/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.076636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.091129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=5287027e82ff49e78cae60565bdcc611, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.092569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.107000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 414/820] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:08 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/5287027e82ff49e78cae60565bdcc611/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:32:09.107838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.165134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.169044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620b72-41c8-428a-b0a9-3f8a65f1eac9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.216157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.216887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.217238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.217609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.218758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:09.219183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.220794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.232138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 407/821] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:09.232138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.233100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd131-1941-4d36-9492-a7617bcdd8b9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.280479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.281462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.281739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.282069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.282693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.285641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:09.296623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:09.297685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 00:32:09.298558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:09.303745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 415/822] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:09.304619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.356717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.364090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99ba79-997e-47b5-bf4b-8315ad6f085d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.401367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.402316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.402749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.403172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.414677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.417225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.434381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3d18a-afa1-436f-9b69-f1b37b9e76d2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:09.434984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:09.436103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:09.437190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:09.437190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:09.437190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:09.438529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 408/823] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:09.439444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.478557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.479261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.479681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.480023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.480731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:09.481102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.483049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.495348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 416/824] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:09.495348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.495691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.556599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29ac9b-7fba-421d-9189-513d482816ae None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.594010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.594952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.595382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.595820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.596714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] RBAC: Adding query filter params (name=network) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:09.597177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.598682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.604132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 409/825] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:09.605094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.627007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.628455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fd21d-2280-4efb-8982-1ea263e29cc6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.674384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.674997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.675334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.675644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.676333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:09.676687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.678138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.692199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 417/826] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:09.693201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.731793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.745953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a848d8-706e-46ae-8c54-84be264295fa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.788355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.789698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.790172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.790659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.792084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.793922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.804879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c9e1f-a6db-4b7d-9f15-a9ba00777be5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:09.825626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 410/827] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:09.827206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.885095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:09.885667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:09.885947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:09.886218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:09.886857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] RBAC: Adding query filter params (type=network) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:09.887166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:09.888737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:09.902334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 418/828] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:09.909974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:09.919197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:09.962421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ddf75-8ae9-4b81-b47e-51f07644ec14 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:10.008665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:10.009675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.010155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.010641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.031512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:10.033646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:10.055511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:10.059931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:10.060900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:10.061331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:10.061404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 411/829] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:10.061473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbe617-439c-46eb-82a9-9a38cdd4f3af None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:10.068552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:10.109037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:10.109666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.110311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.110587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.112748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:10.115075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:10.153963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 419/830] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:09 2022] POST /identity/v3/endpoints => generated 337 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:10.229693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb93a3f-7e89-4349-9347-755561b5dbb3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:10.267116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:10.281711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:10.282635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.282917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.283190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.283921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:10.284392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:10.285899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4dd0c5-d168-42ce-a774-066838d4b800 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:10.300278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 412/831] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:10.308732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:10.378166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671c81-d8b5-46ca-a525-d21003da0714 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.378541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671c81-d8b5-46ca-a525-d21003da0714 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.378886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671c81-d8b5-46ca-a525-d21003da0714 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.380532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 413/832] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:10.388309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.388309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.388309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.407940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:10.409106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:10.442187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dcbc6-b3e3-4d40-bcfa-950ba00eb729 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:10.485922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 414/833] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:10.487383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:10.488025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.488307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.488608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.489350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=4784d64225054ea9a987581a00b6807b) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:10.489697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:10.495509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:10.503637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 420/834] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:10 2022] GET /identity/v3/role_assignments?user.id=4784d64225054ea9a987581a00b6807b&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:10.789190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:10.893150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa6e81-535a-421c-be88-e3c520d1abc6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:10.915623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:10.940735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:10.941687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:10.942164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:10.942626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:10.953910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:10.959019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:10.960539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:10.961367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:10.963977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 415/835] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:10.975528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:10.988720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:11.080679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68429e75-561a-4e31-b60e-47a89f023fce None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:11.115347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:11.116510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.116786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.117053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.117716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:11.118014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:11.119374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:11.128115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 421/836] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:11.134293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.249342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352651b-f15c-4421-b999-be11285d9e83 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:11.285720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:11.286348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.286761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.287032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.297874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:11.299326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d3000a1b-d02a-4cd0-a218-1725b1af5321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:11.314760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:11.315752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:11.316626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:11.316626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:11.316626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:11.316626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 416/837] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:11.322364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.341100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e3fe-49b8-444d-a488-101ab22c3a43 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.341382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e3fe-49b8-444d-a488-101ab22c3a43 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.341648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e3fe-49b8-444d-a488-101ab22c3a43 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.343285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 417/838] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:11.348130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.348618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.348960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.370674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:11.371905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:11.396835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:11.415301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:11.437913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef8c5f-f616-40ee-a079-85f5c7e61125 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:11.450523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 418/839] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 15 00:32:11.474870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:11.476399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.476861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.477341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.478161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:11.478658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:11.480245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:11.493856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 422/840] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:11.501535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.610088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099259f4-5507-4fa2-a972-6ab84a14dda5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:11.646175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:11.647665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.648131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.648576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.656137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:11.657795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:11.669650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:11.670559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 00:32:11.671647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:11.673774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 419/841] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:11.679610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.729015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.749164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:11.802208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc8e77-0011-4557-a244-a9f762fc2fee None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:11.835686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ce7c1-9e7c-4fd6-be03-c4719f030fc2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:11.850142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:11.851307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.851793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.852458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.853769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:11.854262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:11.856422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:11.868487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:11.871845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:11.872475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:11.872747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:11.873013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:11.883249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 423/842] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:11.885509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:11.891596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:11.891733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:11.892640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:32:11.893513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:11.896873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 420/843] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:11.898533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:11.988361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.011275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48af63-f550-4663-88ad-a484c480af23 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.013385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37379a42-ea8c-4972-b8f8-27a3bfccb0e7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.049507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.050454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.051051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.051616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.052713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:12.053362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.054486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.055169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.055313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.055716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.056097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.064565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 421/844] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:12.070296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.128218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.137498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.139053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.198309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8ab00-e9d2-427e-994d-3abca6519eb9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.233625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.234251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.234525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.234793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.245002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.247161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.248970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:12.265375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:12.266577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:12.267700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:12.267700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:12.267700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:12.273349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 422/845] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:12.274846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.283769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 424/846] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 00:32:12.285515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.391944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.400697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becffca7-52a0-4737-8e21-7f52afaebec7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.410759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e306f82-0485-4854-9f14-53a678b13263 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.449966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.451108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.451632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.452108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.453851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.454458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.454833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.455018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.455666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:12.455965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.457336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef496deb-43b2-4238-a112-73b3e7faacbb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.465818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.467721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.469373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 423/847] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:12.470571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be775b-2a1f-45dd-9733-16d02ad9bd89 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.470851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be775b-2a1f-45dd-9733-16d02ad9bd89 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.471185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be775b-2a1f-45dd-9733-16d02ad9bd89 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.472543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 424/848] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:12.476505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be775b-2a1f-45dd-9733-16d02ad9bd89 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-3ef845a6-2505-44fb-8eb8-185f6acc5319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:12.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:12.488943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:12.489899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:12.489899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:12.489899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:12.489899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 425/849] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:12.490148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.490606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.490606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.510422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:12.511665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:12.536834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.583519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 426/850] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:12.583519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.602268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be775b-2a1f-45dd-9733-16d02ad9bd89 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.653201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.654177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.654856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.655278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.665136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.667722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.679921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:12.684019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:12.685048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:12.685993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:12.690358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 425/851] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:12.691192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.708020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4651619-d672-4b37-9149-98d73f6eb3d1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.745473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.746432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.746914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.747395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.748339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:12.748892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.750434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.763280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 427/852] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:12.769622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.799243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fe6b4-b74a-4658-ae59-27cdabe92eb7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.812869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.836018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.836663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.836988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.837346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.838035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:12.838335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.839833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.858967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 426/853] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:12.859955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.884275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec4720-15d1-479b-993a-a32be012c758 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.919193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.919766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.920041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.920308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.926637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.928089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:12.929761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:12.940011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:12.941042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 428/854] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:12.942409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:12.962541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a33ec-aaee-43a6-8a09-47b62060ad90 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:12.995365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:12.996036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:12.996314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:12.996585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:12.997169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:12.998523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-9a6cb03c-ffda-4efe-be71-df169ea09fb2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:13.007600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:13.009013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:13.009861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 427/855] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:13.010667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416a9bf-e66c-4ed4-bcc2-203af8aa89ad None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.010748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416a9bf-e66c-4ed4-bcc2-203af8aa89ad None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.011078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416a9bf-e66c-4ed4-bcc2-203af8aa89ad None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.012418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 428/856] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:13.014256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416a9bf-e66c-4ed4-bcc2-203af8aa89ad None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.043865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7bd0-8f4b-4a6a-8e5d-116ac7baab3f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.050795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.078181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdac2ac-5bb7-45b8-ac5b-a3caad269f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.078791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdac2ac-5bb7-45b8-ac5b-a3caad269f8d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.079065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdac2ac-5bb7-45b8-ac5b-a3caad269f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.079331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdac2ac-5bb7-45b8-ac5b-a3caad269f8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.116229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416a9bf-e66c-4ed4-bcc2-203af8aa89ad None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.141217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.144985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdac2ac-5bb7-45b8-ac5b-a3caad269f8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.146631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdac2ac-5bb7-45b8-ac5b-a3caad269f8d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.149626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.150208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.150486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.150758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.151444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:13.151781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.153201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.165226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 429/857] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:13.166009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.242641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 429/858] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:13.243837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.244119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.244391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.259220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.262363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:13.263693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:13.266544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052e4a4-1714-404b-a316-8b3e13868fb4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.301279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.301877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.302154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.302428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.303512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:13.303825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.305194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.309565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 430/859] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:13.310329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.328548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 430/860] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:13.329610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.356657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.432289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be12f5d-e333-4416-b5ae-65080ee94f62 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.450122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5969e-2799-4c4b-9ce5-836b07f113fb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.472898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.473499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.473771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.474038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.474753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=0587d0fb93ad478fa931141ac9e1a5de) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:13.475075 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.476552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.490560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 431/861] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/role_assignments?user.id=0587d0fb93ad478fa931141ac9e1a5de&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:13.490560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.505174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.505743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.506020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.506289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.511512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.524934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.526422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:13.547545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:13.548576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:13.549612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:13.549612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:13.549612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:13.549612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 431/862] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:13.555164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.604194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbf8e5-0d1c-479f-8468-84233116303f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.641896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.642511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.642789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.643062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.643763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:13.644070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.645501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.650535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 432/863] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:13.651206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.651390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.687107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db113f4f-1fef-4a37-a546-57df8bdc7f87 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.738722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.739348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.739676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.739949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.740609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:13.741533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.742928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.762046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 432/864] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:13.767789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.786608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.804995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e69b-cf7b-46ba-8fa5-55d9a64077a7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.844444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.845092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.845451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.845720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.847419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.847793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:13.857331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:13.858330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:13.859232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 433/865] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:13.864351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.883786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f970b6-a346-48eb-b6f1-b9795edf6eab None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:13.918757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:13.919816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:13.920305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:13.920763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:13.927796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:13.929599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:13.941170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:13.942130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:13.943041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:13.950170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:13.950747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 433/866] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:13.951860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:13.975584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd7039-5687-4bed-9abd-b2cce40fcead None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:14.010145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:14.010782 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:14.011055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:14.011319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:14.011971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:14.012320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:14.013685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:14.023729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 434/867] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:14.024974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:14.051913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:14.056068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bc3af-5c05-4229-a693-e4c8e27750a9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:14.103816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:14.104444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:14.104717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:14.104997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:14.106524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:14.106835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:14.108291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:14.127515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 434/868] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:14.130201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:14.141038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661ed58-3e50-453a-8c88-441d7a7d80a7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:14.175997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:14.177325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:14.177810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:14.178852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:14.179712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:14.180207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:14.181827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f334c5-e367-4664-a016-652ea821cd04 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:14.187070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 435/869] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:14.208273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:14.253380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac2aa6-3b63-49eb-b809-7a21efcd0b69 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:14.292479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:14.293130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:14.293407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:14.293700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:14.294442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=5287027e82ff49e78cae60565bdcc611) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:14.294748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:14.296239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b5a95-d3f5-4689-b16d-9f301f2339d6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:14.303978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 435/870] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:14 2022] GET /identity/v3/role_assignments?user.id=5287027e82ff49e78cae60565bdcc611&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:15.765739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb802bd-ca30-4179-9fdd-4da65271e78a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:15.766273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb802bd-ca30-4179-9fdd-4da65271e78a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:15.766273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb802bd-ca30-4179-9fdd-4da65271e78a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:15.768052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 436/871] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:15.773928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3bc0a-a16b-48aa-8195-eff208c094f6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:15.774452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3bc0a-a16b-48aa-8195-eff208c094f6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:15.774452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3bc0a-a16b-48aa-8195-eff208c094f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:15.793843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-83d3bc0a-a16b-48aa-8195-eff208c094f6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:15.795037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-83d3bc0a-a16b-48aa-8195-eff208c094f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:15.862610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 436/872] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:15.883274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258add-b640-4ae6-b912-5126061e4693 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:15.883852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258add-b640-4ae6-b912-5126061e4693 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:15.884114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258add-b640-4ae6-b912-5126061e4693 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:15.886699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 437/873] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:15.890883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:15.891171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:15.891445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:15.909322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:15.910495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:15.964244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:15.982094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 437/874] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:16.140604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258add-b640-4ae6-b912-5126061e4693 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:16.221705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:16.244565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258add-b640-4ae6-b912-5126061e4693 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:16.252177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:16.285312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:16.286085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.286360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.286630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.287311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:16.289170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:16.298702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:16.299748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:16.300588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 438/875] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:16.305923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:16.369353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a141d3e-c666-4471-8fe3-9382e4151ecc None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:16.407236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b5f00-bd38-49d4-bff9-f159969732f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:16.407892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5f00-bd38-49d4-bff9-f159969732f8 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.408167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5f00-bd38-49d4-bff9-f159969732f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.408438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b5f00-bd38-49d4-bff9-f159969732f8 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.409103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b5f00-bd38-49d4-bff9-f159969732f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:16.410456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b5f00-bd38-49d4-bff9-f159969732f8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:16.439386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8522-464a-4671-b2d9-69cae48e0127 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:16.439704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 438/876] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:16 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:16.447201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ed62-95e2-4067-a927-371757fe2cf9 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.447470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ed62-95e2-4067-a927-371757fe2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.447834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ed62-95e2-4067-a927-371757fe2cf9 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.453594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 439/877] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:16.454997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41928-c4a8-4b5f-8d0a-36f6013c9bac None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.455276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41928-c4a8-4b5f-8d0a-36f6013c9bac None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.455600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41928-c4a8-4b5f-8d0a-36f6013c9bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.476352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-85d41928-c4a8-4b5f-8d0a-36f6013c9bac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:16.477591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-85d41928-c4a8-4b5f-8d0a-36f6013c9bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:16.489499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:16.490262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.491102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.491390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.492096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:16.492450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:16.494103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:16.504994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 439/878] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:16.506141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:16.527921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:16.537340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:16.581881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 440/879] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:16.602569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77336375-b75e-47c0-bf2b-eea3d0cfa7dd None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.602930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77336375-b75e-47c0-bf2b-eea3d0cfa7dd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.603270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77336375-b75e-47c0-bf2b-eea3d0cfa7dd None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.605013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 441/880] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:16.610473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.610819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.611223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.636480 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:16.638010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:16.653963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef1ca3-1a69-4e50-8988-8dd5e6b034a4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:16.705943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:16.706553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:16.706854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:16.707123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:16.707832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:16.708110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:16.709486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:16.714792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 440/881] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:16.721474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 442/882] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:16.852299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:16.927921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:16.967562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee9adc-4fd1-472a-a17b-9c049ee4ed48 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:17.002783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:17.003735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:17.004164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:17.004580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:17.005376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:17.005658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:17.007300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:17.029258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 441/883] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:16 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:17.126518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680160e-80cf-4e32-8ae7-ce0ea3de5356 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:17.129206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:17.173699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:17.174337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:17.174668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:17.174942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:17.187680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:17.192776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:17.193840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:17.196321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 443/884] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 00:32:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:17.200882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:17.321787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c659fc0-59be-4fda-9d43-01b7136204f1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:17.348081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:17.358746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:17.358746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:17.358746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:17.358746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:17.359500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:17.359500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:17.360818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:17.372441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 442/885] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:32:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:17.377888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:17.492362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f7d1-b939-4961-8d73-db1eb5189802 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:17.527652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:17.528278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:17.528549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:17.528812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:17.539700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:17.541135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:17.556593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:17.558887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:17.561171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:17.561171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:17.561171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:17.561171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 444/886] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:17.567503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:17.667767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6d613-3e3d-408b-b178-481272e9367c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:17.708491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:17.709548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:17.710048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:17.710537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:17.711403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:17.711971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:17.713594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:17.727867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 443/887] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:17.738651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:17.749493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:17.843514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882bf06-01b2-4228-8c08-3f75955cbd98 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:17.878385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:17.879012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:17.879283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:17.879571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:17.887446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:17.888895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:17.900497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:17.901445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:17.902405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 445/888] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:32:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:17.907442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:17.955738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:18.007959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1848b-6eff-4229-b9b8-b9839458674c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:18.046216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:18.047194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.047726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.048199 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.049128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:18.049651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:18.051267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:18.069163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 444/889] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 00:32:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:18.077888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:18.149783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:18.180258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc127d1-7c5a-4695-95c0-348186d83b8a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:18.217921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:18.218449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.218722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.218991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.290088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:18.291693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:18.360365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:18.392816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 446/890] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:18 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:18.398603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:18.502267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed48d0-f3b2-40d0-a84f-7e5efa7969df None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:18.511097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:18.538933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:18.539584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.539863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.540136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.550404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:18.551890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:18.568173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:18.569616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:18.570517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:18.570517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:18.570517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:18.570517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 445/891] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:18.575341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:18.684665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295689e7-99a0-4270-b4ff-d128f1989bcb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:18.720337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:18.720940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.721213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.721480 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.722119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:18.722416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:18.723795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88804ae-121a-47c3-9c98-186af20e0131 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:18.735845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 447/892] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:18.741465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:18.787353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a152c25-0342-4200-a726-b71516af77cc None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.787646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a152c25-0342-4200-a726-b71516af77cc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.787899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a152c25-0342-4200-a726-b71516af77cc None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.789245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 448/893] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:18.794503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.794788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.795057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.814885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:18.816117 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:18.844729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:18.864404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c259c-c60e-4d4d-aca1-9e3201699a4d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:18.890354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 449/894] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:18.905864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:18.906495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:18.906772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:18.907042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:18.914763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:18.916239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-aff0e041-c88f-4f88-85ee-110c2e622043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:18.927616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:18.928565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:18.929480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 446/895] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:18.934442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.047979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06074f85-bbe2-4b7e-8863-bf3a9b2fcab5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.083599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.084966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.085192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.085465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.086160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:19.086462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.087914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.100556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 450/896] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:19.106200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0732f9-643f-4562-b7af-f0cb7c6dde27 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.106555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0732f9-643f-4562-b7af-f0cb7c6dde27 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.106925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0732f9-643f-4562-b7af-f0cb7c6dde27 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.108438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.111633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 447/897] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:19.113688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.114178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.114652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.133573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:19.135033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:19.136681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:19.151460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:19.208361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 448/898] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:19.210179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.239522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21cee-810a-48a6-b349-a143d7b0ac55 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.284588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.285467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.285745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.286012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.287426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] RBAC: Adding query filter params (role.id=21bf7ea27a3a45289ac89c0c64ee9feb, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=5287027e82ff49e78cae60565bdcc611) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:19.287764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.289736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3e69f-a983-4304-8cb7-7d5c7c8f68a3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.297251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 451/899] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/role_assignments?user.id=5287027e82ff49e78cae60565bdcc611&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=21bf7ea27a3a45289ac89c0c64ee9feb => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:19.339327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c68f-b61c-4f4a-a0ee-40093fb5e6b5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.367420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec9cf-c46d-4b7c-97a7-fb707c696859 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.367767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec9cf-c46d-4b7c-97a7-fb707c696859 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.368011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec9cf-c46d-4b7c-97a7-fb707c696859 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.369641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 452/900] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:19.377316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.377927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.378437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.378907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.379641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.379920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.380190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.380782 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.382822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:19.394170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:19.395247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:19.396128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 449/901] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:19.402283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.408916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:19.410623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:19.420867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:19.519509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 453/902] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:19.519509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.540460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39d8cf-d013-4087-aad9-2b6ba892b605 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.550773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:19.576308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.576910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.577242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.577513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.578142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:19.578433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.579856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.585606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 450/903] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:19.592340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.641888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190daaa-fbad-411f-9dde-b4578cef60ec None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.688923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.689531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.689804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.690068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.690747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.692199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.700887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:19.718432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 454/904] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:19 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:32:19.733184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc1baa-4ad8-437e-8461-967068faa5dd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.779724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.780403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.781150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.781420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.782143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:19.782448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.784044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.790549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 451/905] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:19.797317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.797681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.901229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:19.934438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bbbfc-e716-4d75-a2ae-acbbae2c5999 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.945817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7becb-7abe-4d71-85d4-3494cbcd2816 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:19.972696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.973335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.973611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.973880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.974468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.975904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:19.985564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:19.986497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:19.987379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 455/906] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:19.994785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:19.995637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:19.996245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:19.996530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:19.996806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:19.997554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] RBAC: Adding query filter params (interface=public, service_id=d9a3ab51ecbf4f25b6765fb7e5c791cb, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:19.997862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:19.999231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:20.002221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:20.005726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 452/907] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/endpoints?service_id=d9a3ab51ecbf4f25b6765fb7e5c791cb&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:20.013526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:20.116973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e5000-ca25-4587-a08e-869478a21a13 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:20.132227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdf971-d113-419a-ab94-80173c77c08a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:20.152495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:20.153231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:20.153671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:20.154010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:20.154667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:20.156195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:20.169094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 453/908] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:20 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:32:20.175062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:20.175963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:20.176379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:20.176785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:20.177569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:20.178571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:20.180365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:20.187751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 456/909] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:20.193288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:20.268365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:20.313191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4d9a-b471-45de-be0b-3408afd25982 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:20.356800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:20.357402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:20.357678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:20.357949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:20.358573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:20.358871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:20.360250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:20.368437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 454/910] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:20.373920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:20.377742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:20.478407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54238ac-9b43-414e-ba9b-8e17641268ca None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:20.513185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:20.513797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:20.514074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:20.514345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:20.515046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] RBAC: Adding query filter params (interface=public, service_id=fec45096eaba44e79e547025b4fd13d5, region_id=RegionOne) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:20.515352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:20.516733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341f724-a068-4304-8ffe-f8a82b67596e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:20.522329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 457/911] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:20 2022] GET /identity/v3/endpoints?service_id=fec45096eaba44e79e547025b4fd13d5&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:20.527925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:20.637385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf032ce7-742a-429a-9262-a790de4b504d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:20.667197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:20.683513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d2a82-4f15-43b7-85ff-ccc934204aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:20.683513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d2a82-4f15-43b7-85ff-ccc934204aed None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:20.683513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d2a82-4f15-43b7-85ff-ccc934204aed None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:20.683513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d2a82-4f15-43b7-85ff-ccc934204aed None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:20.683871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d2a82-4f15-43b7-85ff-ccc934204aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:20.683871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d2a82-4f15-43b7-85ff-ccc934204aed None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:20.688308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 455/912] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:20 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:32:21.513124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392074b-2ba7-4499-8ba1-42381718ff72 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:21.513680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392074b-2ba7-4499-8ba1-42381718ff72 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:21.513680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392074b-2ba7-4499-8ba1-42381718ff72 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:21.515246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 458/913] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:21.520412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:21.520794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:21.520976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:21.539790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:21.540957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:21.610017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 456/914] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:21.887740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392074b-2ba7-4499-8ba1-42381718ff72 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:21.960097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:21.997089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392074b-2ba7-4499-8ba1-42381718ff72 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.035925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5153b-54c5-44e2-90c4-be9eb48bc548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.036913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5153b-54c5-44e2-90c4-be9eb48bc548 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.037448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5153b-54c5-44e2-90c4-be9eb48bc548 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.037910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5153b-54c5-44e2-90c4-be9eb48bc548 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.048996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5153b-54c5-44e2-90c4-be9eb48bc548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-74b5153b-54c5-44e2-90c4-be9eb48bc548 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:22.054051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:22.054967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:22.055841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:22.058366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 459/915] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 00:32:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:22.064026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.064346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61b65-f390-49ea-8a5d-8830bc6b83bc None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.064622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61b65-f390-49ea-8a5d-8830bc6b83bc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.064888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61b65-f390-49ea-8a5d-8830bc6b83bc None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.066369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 460/916] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:22.072874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.073381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.074377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.098717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:22.100718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:22.185293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6568e0-6c80-42fc-9238-084518869477 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.196252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 461/917] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:22.218195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:22.225718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.226325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.226590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.226854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.227562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:22.227866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:22.229280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b140da-4fc8-4d5c-a734-24bc61ed67b6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:22.242210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 457/918] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:22.248286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.315225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:22.360295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d3ce-bea1-49e9-9a7a-f5ae962a84ff None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.388354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620149da-7a1f-4e21-9a59-48ceea4dd93c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.388653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620149da-7a1f-4e21-9a59-48ceea4dd93c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.388911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620149da-7a1f-4e21-9a59-48ceea4dd93c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.391549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 458/919] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:22.395395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.395707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.395980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.400046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.400638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.400915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.401179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.411407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:22.412921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:22.417034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:22.418493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:22.429169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:22.430142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:22.431060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:22.431060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:22.431060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:22.431060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 462/920] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:22.440486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.492558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 459/921] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:22.525002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.568366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed88b2a-ec54-465f-9155-7e6cf1fa5c82 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.582448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:22.616011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.617383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.618273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.619070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.619948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:22.620662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:22.623006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:22.640102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231e52e-a1bc-4885-b355-e77f6bb52950 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.640603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 463/922] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:22.648038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.665621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:22.691979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.692792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.693324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.693795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.694626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:22.696203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:22.705341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:22.706327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:22.707247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:22.710547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 460/923] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:22.716477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.774504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e177-02ac-4dc8-ad94-2a3445fcc19c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.812318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.813089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.813366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.813636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.822438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b4549-c488-4ed9-bf7d-8c3b5d75945f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.824386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:22.828034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:22.845479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:22.846533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 464/924] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:22.853156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.862196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:22.862829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:22.863107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:22.863375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:22.864049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:22.864350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:22.865799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:22.871123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 461/925] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:22.872093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:22.892500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:22.919420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:22.969123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fcbe-c837-477a-9470-91cd1c1e1244 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:22.978316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a3d47-fd56-47e0-aa5f-0ffcf05123e5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.000899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.001516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.001789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.002056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.002633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.004037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.011818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.012418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.012696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.012966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:23.013236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:23.014350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 00:32:23.015349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.016125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:23.016125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.016260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 465/926] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:23.016330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067af1c5-e6af-41c3-bb88-be315b77ba55 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.016406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.035454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 462/927] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 00:32:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:23.036879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a49679-b74d-4a56-bced-b89f5d39de21 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.037272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a49679-b74d-4a56-bced-b89f5d39de21 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.037616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a49679-b74d-4a56-bced-b89f5d39de21 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.039170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 463/928] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:23.040114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a49679-b74d-4a56-bced-b89f5d39de21 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.107274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.113939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a56afa-cd61-4592-b03d-328549959db7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.141164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.146251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a49679-b74d-4a56-bced-b89f5d39de21 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.151137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.151770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.152050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.152320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.152951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:23.153249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.154609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.159403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 466/929] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:23.160272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.180453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.181049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.181328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.181584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.182199 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:23.182493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.183880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47491d1c-c7d1-4062-9264-5c435bed25aa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.188474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 464/930] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:23.189585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.189835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.190102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.208094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:23.209318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:23.260295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14d1eb-21c5-4c97-9738-43abdd19715f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.273998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 465/931] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:23.274838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.296300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.296933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.297209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.297585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.308087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.309670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:23.324646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:23.325601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:23.326522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:23.326522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:23.326522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.326522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 467/932] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:23.326522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.338289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.378893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.388077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18475525-7f07-4e35-8832-c2c3ca44b049 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.425513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.426112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.426386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.426652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.426899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0310f-ab29-4da8-9b36-b81322163f48 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.427343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] RBAC: Adding query filter params (interface=public, service_id=99efa2c4207141d18e99f4dd7abb92bd, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:23.427672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.430434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.441008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 466/933] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/endpoints?service_id=99efa2c4207141d18e99f4dd7abb92bd&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:23.441008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.465018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.465619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.465892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.466161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.466783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:23.467081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.468458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.473234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 468/934] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:23.473887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.559044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08de3-1591-4639-ac18-f3aaadd0b205 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.560088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.584157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49434-52ca-4fc3-b3d8-d8736618a9ce None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.591565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.592166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.592442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.592711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.593347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:23.593647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.595006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.606359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 467/935] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:23.607112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.622385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.623003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.623272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.623572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.624174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.625526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.627845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.633165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 469/936] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:32:23.634624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.736002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af088a-ac59-4604-878e-c5549ca53006 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.753100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b1b14-11ff-4849-a41c-db9d8280bd70 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.773565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.774187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.774466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.774739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.775414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.776887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.792241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:23.793055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:23.793421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:23.793727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:23.794417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:23.795836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:23.801113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 468/937] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:23.801702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:23.816413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:23.817441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:23.818912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 470/938] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:23.825297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:23.831038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.932300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:23.971793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb1a0-36b7-4b09-b86f-33735d0b5156 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:23.973542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39681cb3-c337-4032-823a-460e73e8be21 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:24.017460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:24.018403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:24.018594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:24.019013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:24.019239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:24.019630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:24.019741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:24.020057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:24.020860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:24.021314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:24.023711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:24.028709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:24.029885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 471/939] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:24.030257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:24.035633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:24.042944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:24.043926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:24.044834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:24.049922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 469/940] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:24.051034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:24.121253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:24.169283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae0fa9-e1b7-4f86-9190-45f9d6b2ecc2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:24.177064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2fb3-17b7-441b-aaf8-51dabeaa6871 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:24.213304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:24.213904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:24.214528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:24.214748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:24.214850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:24.215064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:24.215528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:24.215714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:24.216019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:24.216401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:24.217288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:24.217391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:24.217885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:24.219702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:24.220259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:24.227569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 472/941] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:24.229497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:24.241081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 470/942] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:24.246804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:24.344155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a03aed-47b7-45c0-92d9-03f731064d01 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:24.379594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf87d-5b03-4712-a3a3-5daab1486f55 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:24.379898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:24.386828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab19e1-c720-40a0-88ad-c8a6f9da4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:24.387758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab19e1-c720-40a0-88ad-c8a6f9da4b9d None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:24.388260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab19e1-c720-40a0-88ad-c8a6f9da4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:24.389159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab19e1-c720-40a0-88ad-c8a6f9da4b9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:24.390050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab19e1-c720-40a0-88ad-c8a6f9da4b9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:24.392398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab19e1-c720-40a0-88ad-c8a6f9da4b9d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:24.427423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 473/943] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:32:24.433824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf424f-6c8c-437d-8d55-14639fa3ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:24.434508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf424f-6c8c-437d-8d55-14639fa3ffeb None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:24.434785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf424f-6c8c-437d-8d55-14639fa3ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:24.435077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf424f-6c8c-437d-8d55-14639fa3ffeb None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/5287027e82ff49e78cae60565bdcc611/roles/21bf7ea27a3a45289ac89c0c64ee9feb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:24.445994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf424f-6c8c-437d-8d55-14639fa3ffeb None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=5287027e82ff49e78cae60565bdcc611, role_id=21bf7ea27a3a45289ac89c0c64ee9feb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:24.448134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf424f-6c8c-437d-8d55-14639fa3ffeb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:24.466277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 471/944] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:24 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/5287027e82ff49e78cae60565bdcc611/roles/21bf7ea27a3a45289ac89c0c64ee9feb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:32:25.924295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f5635-4b34-4aad-a9dc-baaee83fb32d None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:25.924800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f5635-4b34-4aad-a9dc-baaee83fb32d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:25.924800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f5635-4b34-4aad-a9dc-baaee83fb32d None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:25.926472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 474/945] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:25.931924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8df854-0c53-41b7-89a4-5bdbea67bf25 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:25.932303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8df854-0c53-41b7-89a4-5bdbea67bf25 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:25.932482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8df854-0c53-41b7-89a4-5bdbea67bf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:25.950805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cc8df854-0c53-41b7-89a4-5bdbea67bf25 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:25.952034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cc8df854-0c53-41b7-89a4-5bdbea67bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:26.010443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:26.046748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 472/946] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:26.190692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6168-f5da-4a1f-acda-e8e32f35fc6a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.190692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6168-f5da-4a1f-acda-e8e32f35fc6a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.190692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6168-f5da-4a1f-acda-e8e32f35fc6a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.191375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 475/947] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:26.197522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.198047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.198593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.218329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:26.219909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:26.299070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 473/948] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:26.335184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6168-f5da-4a1f-acda-e8e32f35fc6a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:26.447657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6168-f5da-4a1f-acda-e8e32f35fc6a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:26.483326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:26.483995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.484272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.484541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.485132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:26.486686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:26.498343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:26.499393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.500448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 476/949] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:26.509175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:26.613389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93071775-d4e6-455f-b8e9-99823071fe6c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:26.618058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:26.621422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:26.659553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:26.660253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.660536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.660808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.661592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:26.661899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:26.663540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:26.669504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 474/950] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:26.677404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:26.686592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:26.759417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d230694-8b4b-4f1f-b9b5-6d1e74937ce2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:26.795351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e125217-5d4e-4075-ad9c-de06b0800c5a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:26.797578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:26.798192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.798472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.798748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.799417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:26.800871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:26.803540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING keystone.server.flask.common [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 00:32:26.823199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.sql.core [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 00:32:26.823199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 00:32:26.823199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [parameters: {'id': 'fcbf51cdcc8f4c7f89f4fac45e8262e9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 00:32:26.823199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81609) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 00:32:26.832535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:26.833139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e5838f9f-0f46-4dca-8edd-9c65f8b3088d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fcbf51cdcc8f4c7f89f4fac45e8262e9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fcbf51cdcc8f4c7f89f4fac45e8262e9. Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application conn.query(q) Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 00:32:26.833325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result.read() Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 15 00:32:26.834801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application next(self.gen) Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application yield resource Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application next(self.gen) Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise self.value Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application session.commit() Sep 15 00:32:26.836008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.session.flush() Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application compat.raise_( Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 00:32:26.837116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 00:32:26.838746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application conn.query(q) Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application result.read() Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 00:32:26.840438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [parameters: {'id': 'fcbf51cdcc8f4c7f89f4fac45e8262e9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:26.842785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:26.844684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fcbf51cdcc8f4c7f89f4fac45e8262e9. Sep 15 00:32:26.853647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:26.854823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.854823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.854823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:26.854823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:26.854823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:26.855583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 477/951] 10.222.0.15 () {60 vars in 1140 bytes} [Thu Sep 15 00:32:26 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:26.855583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749f646-5f0f-45ac-a513-08153fc438fb None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:26.855583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749f646-5f0f-45ac-a513-08153fc438fb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:26.855583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749f646-5f0f-45ac-a513-08153fc438fb None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:26.856163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 475/952] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:26.856163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:26.856435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 478/953] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:26.856435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749f646-5f0f-45ac-a513-08153fc438fb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:26.888623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:26.965289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749f646-5f0f-45ac-a513-08153fc438fb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:26.965966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a349-3a58-4803-ae34-78475d1a3ed1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:26.973035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:27.004273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3e497-6a97-4569-92ea-08e4454ff551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.004874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3e497-6a97-4569-92ea-08e4454ff551 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.005149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3e497-6a97-4569-92ea-08e4454ff551 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.005415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3e497-6a97-4569-92ea-08e4454ff551 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.006113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3e497-6a97-4569-92ea-08e4454ff551 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:27.007610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3e497-6a97-4569-92ea-08e4454ff551 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:27.007925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.008809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.009084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.009355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.015969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:27.017321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:27.027584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 479/954] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:27.028857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aed27-69e6-4495-841e-b40f402a91bd None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.029147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aed27-69e6-4495-841e-b40f402a91bd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.029413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aed27-69e6-4495-841e-b40f402a91bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-595872a5-375a-4949-8d31-6be5c6dc8f58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:27.029510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:27.030437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:27.031419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 476/955] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:27.036958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccebf42-3456-4c28-9dab-a43a1da678ae None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.037823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccebf42-3456-4c28-9dab-a43a1da678ae None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.038177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccebf42-3456-4c28-9dab-a43a1da678ae None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.039630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 477/956] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:27.040237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccebf42-3456-4c28-9dab-a43a1da678ae None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.058126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7d4aed27-69e6-4495-841e-b40f402a91bd None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:27.059877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7d4aed27-69e6-4495-841e-b40f402a91bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:27.148373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:27.153353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 480/957] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:27.154962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.155509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.155992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.167281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccebf42-3456-4c28-9dab-a43a1da678ae None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:27.178424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:27.180010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:27.207832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.208589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.209470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.209894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.210663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:27.210970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:27.212434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:27.233153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 478/958] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:27.259960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 481/959] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:27.441643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.549252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.552297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40050379-9050-4c71-ac85-cfb6d0098aae None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:27.576716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:27.597229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.598046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.598454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.598819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.609939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:27.614953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:27.616663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:27.617790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:27.619677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 479/960] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:27.626628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.651079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:27.669078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d363832-b9cd-4248-81ff-f67ef096282d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:27.710786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.711834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.712260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.712797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.713563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:27.715338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:27.725294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:27.726375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:27.727238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:27.730265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 482/961] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:27.736304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.765747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fdf4e-d625-4ec5-a8d6-618b865136e7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:27.817395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.819015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.819756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.820454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.821378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:27.822488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:27.825016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:27.831131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:27.839352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 480/962] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:27.844701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.859017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7c7c2-a938-4875-a3e0-fa60a0ceb21c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:27.896968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:27.897980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:27.898465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:27.898940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:27.899841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:27.900347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:27.901924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:27.911908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 483/963] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:27.914437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:27.939936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:27.966870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbec9f-d989-44a9-97ae-64e5fde4213a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:28.006799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:28.007451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:28.007762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:28.008037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:28.022572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:28.023809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81214fb5-d251-43d2-ac6c-c76f457a488c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:28.024264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:28.039922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:28.041032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:28.041942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:28.041942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:28.041942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:28.041942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 481/964] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:28.046921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:28.064292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:28.065402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:28.066045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:28.066521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:28.067755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] RBAC: Adding query filter params (type=compute) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:28.069018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:28.070729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:28.084530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 484/965] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:28.169264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14bf12-3c27-4d88-a3a8-76133eeec7b2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:28.198607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:28.208278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:28.209488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:28.209778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:28.210050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:28.210704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:28.211007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:28.212421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:28.230538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 482/966] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:28.232163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:28.247185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:28.348636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e977f1-91ea-4595-a3c6-ecaab1720bec None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:28.383077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:28.383985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:28.384266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:28.384512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:28.392110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:28.393527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:28.405330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:28.406319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:28.407323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 485/967] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:32:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:28.412888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:28.514716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6a83-c74b-4e51-8218-6a23bec1126f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:28.554449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:28.555405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:28.555921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:28.556398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:28.557337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:28.557929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:28.559611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:28.577598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 483/968] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 00:32:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:28.587512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:28.659658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:28.693881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a755d-c186-46d7-8e5d-26ebb418d0eb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:28.729046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:28.729656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:28.729931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:28.730261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:28.799271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:28.800744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:28.873013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:28.911344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 486/969] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:28 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:28.913008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:28.961006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:29.020617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2569bd-0108-4930-ab51-e8d453546c83 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:29.059149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:29.059717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.059997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.060271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.071512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:29.072979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:29.088437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:29.089473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:29.090390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:29.090390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:29.090390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:29.090390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 484/970] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:29.095419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:29.195790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c507ec4-f090-4242-b07b-fb5220bdf5ca None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:29.229867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:29.230478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.230946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.231215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.231913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:29.232213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:29.233622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6877b-1ee0-4e2a-a112-d43fe7282e3f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:29.251809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 487/971] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:29.254201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:29.360657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:29.365620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff396e-13da-4eef-9160-59c9cd9bbaad None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:29.368098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa985c-c3fa-4f1a-b38d-6161e76de0f3 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.368475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa985c-c3fa-4f1a-b38d-6161e76de0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.368829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa985c-c3fa-4f1a-b38d-6161e76de0f3 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.370394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 488/972] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:29.375350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.375748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.376271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.395492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:29.397007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:29.411215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:29.412589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.413221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.413784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.422548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:29.424081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:29.426259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:29.443978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:29.444936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:29.445822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:29.448673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 485/973] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:29.453861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:29.485059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 489/974] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:29.564772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56980278-995d-49f5-9856-ae81ecc7ff97 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:29.598837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:29.600428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.600929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.601451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.602414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:29.602946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:29.604723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deafcf8-9e38-4a38-8818-9a0edc4158d7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:29.620354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 486/975] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:29.631529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:29.731557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193e635-fa6a-493e-8b20-526fe748122b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.731883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193e635-fa6a-493e-8b20-526fe748122b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.732369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193e635-fa6a-493e-8b20-526fe748122b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.734517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 487/976] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:29.741360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.741731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.741944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.745401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c60515-ac5d-4e08-9315-db215b16d0e8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:29.763146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:29.764756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:29.780949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:29.785178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:29.786016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.786541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.786736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.787456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] RBAC: Adding query filter params (role.id=21bf7ea27a3a45289ac89c0c64ee9feb, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=5287027e82ff49e78cae60565bdcc611) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:29.787805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:29.789535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:29.796142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 490/977] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:29 2022] GET /identity/v3/role_assignments?user.id=5287027e82ff49e78cae60565bdcc611&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=21bf7ea27a3a45289ac89c0c64ee9feb => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:29.796906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:29.825791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:29.875439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 488/978] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:29.918517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5c53e-e40d-444c-a301-f71c056d578e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:29.957448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:29.958576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:29.959615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:29.960090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:29.960889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:29.962613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:29.972639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:29.973710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:29.975134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:29.977626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 491/979] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:29.984204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.092489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51874c-e56f-462a-8eed-22c45321d0ae None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.127592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.128125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.128398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.128662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.129289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:30.129587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.130976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.136555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 489/980] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:30.142150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.155802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.171685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:30.263503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:30.276903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655dace-cd44-4504-a638-3ba556a06c76 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.280368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c5905-fafb-4b4c-9739-6506f4769d4e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.319292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.320403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.320866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.321017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.321624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.321778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.322206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.322575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.322883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.323392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:30.323818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.324829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.325370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2181764-ed5c-48a6-a3a0-932e7e87aac9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.328494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING keystone.server.flask.common [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 00:32:30.331102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 492/981] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:30.332421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4f94-b2dd-4c6e-b41c-394f62635bae None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.332801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4f94-b2dd-4c6e-b41c-394f62635bae None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.333173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4f94-b2dd-4c6e-b41c-394f62635bae None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.335412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 493/982] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:30.337526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee4f94-b2dd-4c6e-b41c-394f62635bae None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.344538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.sql.core [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 00:32:30.344538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 00:32:30.344538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [parameters: {'user_id': 'f7f1b899642e4d1db03e1c06ac9ca20d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 00:32:30.344538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81608) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b9776887-5b55-4820-9c32-30df05111a37 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application conn.query(q) Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result.read() Sep 15 00:32:30.363311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 00:32:30.364426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application next(self.gen) Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application yield resource Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application next(self.gen) Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise self.value Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application session.commit() Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 00:32:30.365477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.session.flush() Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application compat.raise_( Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 00:32:30.366393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 00:32:30.367297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application conn.query(q) Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result.read() Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 00:32:30.370850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f7f1b899642e4d1db03e1c06ac9ca20d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:30.374084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 15 00:32:30.375963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 490/983] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 00:32:30 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.378255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.387152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:30.388834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:30.460008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 491/984] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:30.460198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee4f94-b2dd-4c6e-b41c-394f62635bae None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.468402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.473065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:30.499514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.500187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.500875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.501150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.501987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] RBAC: Adding query filter params (interface=admin, service_id=99efa2c4207141d18e99f4dd7abb92bd, region_id=RegionOne) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:30.502474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.504020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.511834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 494/985] 10.222.0.15 () {56 vars in 1249 bytes} [Thu Sep 15 00:32:30 2022] GET /identity/v3/endpoints?service_id=99efa2c4207141d18e99f4dd7abb92bd&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:30.517074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.543776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:30.605192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3680ed5-9b96-4d58-8ecd-2a19a2307a87 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.649417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.649621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31946029-32e5-496a-86e5-9062d7fe9535 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.650509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.651089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.651586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.659563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.661559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:30.674514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:30.675420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 00:32:30.676393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:30.679629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 492/986] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:30.684806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.687211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.688239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.688517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.688791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.689403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.690760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.703520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 495/987] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:30 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:32:30.752044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:30.766320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:30.816967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29d06b-1041-4499-a00a-e8f71d95d22f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.855968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:30.863078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.864515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.865006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.865537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.866408 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:30.866714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.869036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961cdf-ca48-41a4-b3fb-60a16edb979d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.877271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4562-1b07-499f-98f4-11d29b8c3c3f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:30.915377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 493/988] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 00:32:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:30.916213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472b859-8f32-4f1a-9009-641814b5ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:30.916827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472b859-8f32-4f1a-9009-641814b5ca55 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:30.917108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472b859-8f32-4f1a-9009-641814b5ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:30.917383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472b859-8f32-4f1a-9009-641814b5ca55 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:30.918041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472b859-8f32-4f1a-9009-641814b5ca55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:30.919426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472b859-8f32-4f1a-9009-641814b5ca55 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:30.936833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 496/989] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:30 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:32:32.083297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284239a2-11d5-4fd1-9249-57868d68ff77 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.084410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284239a2-11d5-4fd1-9249-57868d68ff77 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.084410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284239a2-11d5-4fd1-9249-57868d68ff77 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.085799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 494/990] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:32.090920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.091306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.091474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.109374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:32.110562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:32.183717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 497/991] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:32.450942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284239a2-11d5-4fd1-9249-57868d68ff77 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:32.553281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284239a2-11d5-4fd1-9249-57868d68ff77 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:32.575841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:32.588584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:32.589222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.589502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.589827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.590889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:32.592770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:32.603702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:32.604780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:32.605660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:32.607636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 495/992] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 15 00:32:32 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:32.612399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:32.686446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:32.722919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7326d-4868-4051-9f9c-9b6944e02aeb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:32.759791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:32.760398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.760677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.760949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.761582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] RBAC: Adding query filter params (name=image) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:32.761886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:32.763248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378469ad-f77e-40b1-92bb-3ac130fe3210 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:32.772926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 498/993] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:32.773887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:32.880106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce696d-930f-4e81-9802-51e5c1f6f6f0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:32.914757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afe6c8-abfc-498c-96dc-fc8ad882fc60 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.915139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afe6c8-abfc-498c-96dc-fc8ad882fc60 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.915312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afe6c8-abfc-498c-96dc-fc8ad882fc60 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.916866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 499/994] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:32.921688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd375bec-e45a-42be-843a-b3720ddc041f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.921970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd375bec-e45a-42be-843a-b3720ddc041f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.922246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd375bec-e45a-42be-843a-b3720ddc041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.926441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:32.927090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:32.927397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:32.927684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:32.928308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:32.928610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:32.929962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e090e-da0e-474e-9823-ad329d13bad9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:32.943536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 496/995] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:32.943748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bd375bec-e45a-42be-843a-b3720ddc041f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:32.943748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bd375bec-e45a-42be-843a-b3720ddc041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:33.018620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 500/996] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:33.143909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79333da-fa16-4cc4-a685-f217e887e8b6 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.144401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79333da-fa16-4cc4-a685-f217e887e8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.144478 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79333da-fa16-4cc4-a685-f217e887e8b6 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.146042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 497/997] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:33.151688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.152126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.152294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.171102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:33.172307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:33.229186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:33.241573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 501/998] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:33.289456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79333da-fa16-4cc4-a685-f217e887e8b6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.342211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:33.397029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79333da-fa16-4cc4-a685-f217e887e8b6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:33.430546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:33.431233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.431526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.431826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.432412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:33.433764 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:33.442625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:33.444170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 00:32:33.445023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:33.447831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 498/999] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 00:32:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:33.450291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.512176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.556049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c7793-43bd-459f-8bd9-b969f8a3f152 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:33.592535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:33.593552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.593832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.594101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.595089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:33.595396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:33.597140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:33.602014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 502/1000] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:33.608480 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.622840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3970b5-3146-4f6d-b9ae-0449f943bb18 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:33.638482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:33.660572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:33.661193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.661461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.661754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] PATH_INFO: `/v3/services/compute` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.662340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:33.663728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:33.672706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:33.673754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:33.674599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 499/1001] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:33.679935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:33.680143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.731063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3ebd-de1b-40fa-a5ee-deae4e7dcd1b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:33.771028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:33.771697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.771976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.772247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.773239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:33.773239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:33.774511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:33.783780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 503/1002] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 00:32:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:33.784730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.790399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa094f30-2680-4835-a388-c772a2e7d2ac None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:33.829270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:33.830603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.831094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.831860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.832722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:33.833379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:33.836274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:33.843567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 500/1003] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:33.849072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:33.889465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdaaf3-5dd5-445a-8070-7b2dc765aa84 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:33.894360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:33.910670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:33.929411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:33.930238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:33.930720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:33.931164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:33.932052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:33.933501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:33.952143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 504/1004] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:33.958158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63fe81-b440-44c7-b53c-a9261a0df5b6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:34.005195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:34.006644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:34.007064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:34.007472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:34.008613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:34.009063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:34.010672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:34.017164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 501/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:34.023460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:34.132597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe869a2-eee2-4e64-aae3-13d0b5ae12a5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:34.167871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:34.168475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:34.168744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:34.169018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:34.169720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] RBAC: Adding query filter params (interface=public, service_id=94b9790d615f4060acc0cb295a8be257, region_id=RegionOne) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:34.171145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:34.171386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74120614-6331-4ffc-a082-7c829348b90e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:34.176776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 505/1006] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:34 2022] GET /identity/v3/endpoints?service_id=94b9790d615f4060acc0cb295a8be257&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:34.182569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:34.285900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0640a-db80-4745-9e3b-65ea242a609e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:34.287506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:34.321656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922945f-f41d-44bd-9f48-efee94095ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:34.322262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922945f-f41d-44bd-9f48-efee94095ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:34.322537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922945f-f41d-44bd-9f48-efee94095ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:34.322805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922945f-f41d-44bd-9f48-efee94095ad9 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:34.323403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922945f-f41d-44bd-9f48-efee94095ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:34.324801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922945f-f41d-44bd-9f48-efee94095ad9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:34.339388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 502/1007] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:34 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:32:35.144201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474880a9-3933-430d-b1d8-4854e1b682bf None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:35.144620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474880a9-3933-430d-b1d8-4854e1b682bf None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:35.144798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474880a9-3933-430d-b1d8-4854e1b682bf None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:35.146288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 506/1008] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:35.151271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaab15b-3df6-4c30-86da-bab50c5dac34 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:35.151595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaab15b-3df6-4c30-86da-bab50c5dac34 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:35.151876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaab15b-3df6-4c30-86da-bab50c5dac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:35.170039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3aaab15b-3df6-4c30-86da-bab50c5dac34 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:35.171222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3aaab15b-3df6-4c30-86da-bab50c5dac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:35.241365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 503/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:35.516138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474880a9-3933-430d-b1d8-4854e1b682bf None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:35.564513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:35.624124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474880a9-3933-430d-b1d8-4854e1b682bf None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:35.662124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b44b4-3c2f-4aa0-b474-61e54a044c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:35.662728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b44b4-3c2f-4aa0-b474-61e54a044c23 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:35.663002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b44b4-3c2f-4aa0-b474-61e54a044c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:35.663270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b44b4-3c2f-4aa0-b474-61e54a044c23 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:35.663961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b44b4-3c2f-4aa0-b474-61e54a044c23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:35.665311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b44b4-3c2f-4aa0-b474-61e54a044c23 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:35.683561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 507/1010] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:35 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:36.120825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b01c3b-6747-4c66-af94-c39f93eb40f4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:36.121173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b01c3b-6747-4c66-af94-c39f93eb40f4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:36.121414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b01c3b-6747-4c66-af94-c39f93eb40f4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:36.122799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 504/1011] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:36.128316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360e9c3-59db-455d-8beb-d2546e4b47eb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:36.128600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360e9c3-59db-455d-8beb-d2546e4b47eb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:36.128870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360e9c3-59db-455d-8beb-d2546e4b47eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:36.147881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9360e9c3-59db-455d-8beb-d2546e4b47eb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:36.149052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9360e9c3-59db-455d-8beb-d2546e4b47eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:36.224644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 508/1012] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:36.494133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b01c3b-6747-4c66-af94-c39f93eb40f4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:36.508563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf0056-a8e8-4561-ac61-071f547203c1 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:36.508848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf0056-a8e8-4561-ac61-071f547203c1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:36.509125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf0056-a8e8-4561-ac61-071f547203c1 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:36.510468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 509/1013] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:36.515536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:36.515818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:36.516085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:36.534173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:36.535463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:36.561773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:36.598816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:36.607344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b01c3b-6747-4c66-af94-c39f93eb40f4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:36.613198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 510/1014] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:36.644071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:36.644071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:36.644071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:36.644406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:36.647447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:36.648900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:36.650607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 505/1015] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:36.656253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:36.760502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f442360-3368-41f9-9684-895a182d08e4 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:36.794733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:36.795332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:36.795644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:36.795915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:36.796567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:36.797972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:36.831509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 511/1016] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:32:36 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:36.884439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:36.994456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d30ab8-001a-4239-9773-395e9f0f19f6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:37.032527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:37.033081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:37.033358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:37.033628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:37.034208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:37.035597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:37.045725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:37.047006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:37.048263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 506/1017] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:37.053797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:37.067235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:37.155104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f93e0-fd46-4fff-ab86-471a76a1a635 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:37.187094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:37.187715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:37.187987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:37.188253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:37.188873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:37.189164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:37.190512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:37.195739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 512/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:37.201632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:37.231929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:37.306893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38529000-5447-47c4-a962-97c7193e8338 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:37.341193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:37.341805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:37.342084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:37.342469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:37.343111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:37.343489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:37.344870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dd274-4777-4f5d-b70b-2e18cceef185 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:37.354415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 507/1019] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 00:32:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:37.355691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:37.435744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:37.461803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7aa6-2742-4a29-abb1-a77a0ffd09fd None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:37.495101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df787400-d7f5-4167-b0c9-cdd85e5f02af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:37.495728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df787400-d7f5-4167-b0c9-cdd85e5f02af None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:37.496171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df787400-d7f5-4167-b0c9-cdd85e5f02af None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:37.496441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df787400-d7f5-4167-b0c9-cdd85e5f02af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:37.497095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df787400-d7f5-4167-b0c9-cdd85e5f02af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:37.498470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df787400-d7f5-4167-b0c9-cdd85e5f02af None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:37.527521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 513/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:37.930704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e1f0-77f5-4dd9-a927-a1026cad62a2 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:37.931209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e1f0-77f5-4dd9-a927-a1026cad62a2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:37.931209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e1f0-77f5-4dd9-a927-a1026cad62a2 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:37.932798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 508/1021] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:37.938131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:37.938549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:37.938753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:37.957306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:37.959030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:38.032830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 514/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:38.297925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e1f0-77f5-4dd9-a927-a1026cad62a2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:38.392278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:38.403512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e1f0-77f5-4dd9-a927-a1026cad62a2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:38.438234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:38.438857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:38.439130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:38.439395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] PATH_INFO: `/v3/services/image` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:38.439999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:38.441361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:38.450551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:38.451550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:38.452442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:38.455551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 509/1023] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 15 00:32:38 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:38.457838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:38.514815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:38.560229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d086d-a3c4-4d76-b2ad-30d3510f88f5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:38.592452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:38.593047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:38.593321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:38.593588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:38.594207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] RBAC: Adding query filter params (name=image) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:38.594503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:38.595992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:38.601189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 515/1024] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:38.606803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:38.710907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f0448-a7a7-4f56-9014-af40b4699c7b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:38.746315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:38.746918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:38.747191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:38.747456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:38.748123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:38.748427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:38.749785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:38.759996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 510/1025] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:38.761447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:38.863866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36967e15-3c21-4270-a39e-7756a110d19b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:38.884056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:38.899801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:38.900406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:38.900683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:38.900952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:38.901651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] RBAC: Adding query filter params (interface=public, service_id=cbed0a843801421591d2b15b21cf2857, region_id=RegionOne) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:38.902019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:38.903434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ca7a1-b283-41fd-8c5d-a32044022ed0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:38.913596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 516/1026] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:32:38 2022] GET /identity/v3/endpoints?service_id=cbed0a843801421591d2b15b21cf2857&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:38.915192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:38.936702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:39.020694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bd017-597e-4b7a-a6d3-cbe9cd04f2b1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:39.059502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921790-f473-414a-9055-1450ee291682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:39.060138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921790-f473-414a-9055-1450ee291682 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.060415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921790-f473-414a-9055-1450ee291682 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.060682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921790-f473-414a-9055-1450ee291682 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.061282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26921790-f473-414a-9055-1450ee291682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:39.062889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26921790-f473-414a-9055-1450ee291682 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:39.079586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 511/1027] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:32:38 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:32:39.122905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdc1f9-af0c-45df-99a7-15447cf5c644 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.123407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdc1f9-af0c-45df-99a7-15447cf5c644 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.123407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdc1f9-af0c-45df-99a7-15447cf5c644 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.124770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 517/1028] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:39.138380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.138706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.138933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.158722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:39.159973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:39.234716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 512/1029] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:39.505794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdc1f9-af0c-45df-99a7-15447cf5c644 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:39.608982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdc1f9-af0c-45df-99a7-15447cf5c644 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:39.644016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9aa1-5f63-4941-9732-af8f90356970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:39.644244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9aa1-5f63-4941-9732-af8f90356970 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.646527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9aa1-5f63-4941-9732-af8f90356970 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.646527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9aa1-5f63-4941-9732-af8f90356970 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.646527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b9aa1-5f63-4941-9732-af8f90356970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:39.648927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b9aa1-5f63-4941-9732-af8f90356970 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:39.650106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 518/1030] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:39.654593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:39.672485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:39.763587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1af044-f135-4a98-96b6-58fd412ed7fa None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:39.796858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:39.797465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.797737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.798002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.798653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:39.800128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:39.804170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e4857-1e28-4d39-b2db-0718863e7792 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.804457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e4857-1e28-4d39-b2db-0718863e7792 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.804734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e4857-1e28-4d39-b2db-0718863e7792 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.806193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 519/1031] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:39.810848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:39.811124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:39.811383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:39.824147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 513/1032] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:32:39 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:39.838309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:39.839573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:39.894173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:39.917054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 520/1033] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:40.183836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:40.253624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:40.290934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12026551-cdfb-4f08-8873-a7a89a775c37 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:40.325106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:40.325736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:40.326015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:40.326282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:40.337637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:40.342495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:40.343430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:40.344317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 514/1034] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 00:32:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:40.351280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:40.450006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ade4e-71bb-40f0-8c5b-628053d097d2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:40.482841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:40.483443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:40.483770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:40.484040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:40.484700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:40.485001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:40.486345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:40.494861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 521/1035] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:32:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:40.500480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:40.602887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d64f6-c980-4910-a092-de16c0216461 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:40.630928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:40.638149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:40.638791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:40.639066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:40.639336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:40.651350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:40.652844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:40.667696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:40.668685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:40.678349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:40.678349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:40.678349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:40.678349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 515/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:40.681416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:40.747525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:40.809723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3bb82-bcda-4235-ac68-0e493f6d2907 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:40.844250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:40.844906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:40.845186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:40.845454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:40.846093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:40.846393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:40.847775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:40.865506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 522/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:40.866691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:40.973610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108d1e0-0ecd-4d62-9466-5102a4bb4340 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.009968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.010601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.010876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.011184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.018958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.020429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:41.032309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:41.033269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:41.039064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 516/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:32:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:41.039718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.157052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a456dfc-4268-4950-9037-1d7a37049538 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.159418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:41.193653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.194254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.194531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.194846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.195757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:41.196193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.197613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6f50-a48f-4943-bae8-d3894790ecc5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.219509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 523/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:41.220643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.231958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a319af-e598-4532-90e8-9479cc9b93d5 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.232255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a319af-e598-4532-90e8-9479cc9b93d5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.232507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a319af-e598-4532-90e8-9479cc9b93d5 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.233854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 524/1040] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:41.239027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.239299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.240088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.258666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:41.259878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:41.268172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:41.327516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 525/1041] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:41.328118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1a925-8eb2-45d9-8aa4-801abd3701bc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.363423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.364050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.364325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.365113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.432506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.433942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.474822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:41.537882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 517/1042] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:41.539031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.596521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.635703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:41.648567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bfe04-7e97-45c3-ab22-837ba2fe1487 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.686673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.687731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.688011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.688428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.698568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.700029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.709848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daac05-c38b-4cde-b86b-dac28a3242ef None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:41.714708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:41.715862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:41.716927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:41.716927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:41.716927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:41.716927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 526/1043] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:41.721314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.749083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.749704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.749977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.750431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] PATH_INFO: `/v3/services/image` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.750813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.752191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:41.761113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:41.762099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:41.763737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 518/1044] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:41.768069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.777374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:41.828215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119f6a-d385-4dc4-a4f5-804a87f63b60 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.863287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.863918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.864194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.864485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.865136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:41.865434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.866790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.873863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef568c6f-2e38-435a-8828-60eae28a5838 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:41.879547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 527/1045] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:41.885278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.889528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:41.913112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:41.913713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:41.913988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:41.914256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:41.914883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] RBAC: Adding query filter params (name=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:41.915180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:41.916582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:41.923631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 519/1046] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:41.927312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:41.995667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54319e-d2d8-4965-b6df-4122cdadac17 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.001856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:42.030482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.031107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.031382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.031682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.036973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342500f-506a-4b0a-bb32-5d51f409bb01 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.038775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.040239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-2a5bf7c4-2f3a-4fee-8038-285682464a74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:42.051175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:42.052290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:42.053560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 528/1047] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:42.056817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882447-2b90-4fdf-a08b-a98020e828fd None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.057105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882447-2b90-4fdf-a08b-a98020e828fd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.057385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882447-2b90-4fdf-a08b-a98020e828fd None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.058664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 529/1048] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:42.059355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed882447-2b90-4fdf-a08b-a98020e828fd None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:42.077148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.077751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.078031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.078348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.078933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] RBAC: Adding query filter params (type=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:42.079238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.080648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca72a4c-13f7-471f-b5e1-31f9746db22f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.085990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 520/1049] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:42.087109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.087390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.087692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.106125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:42.107321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:42.124672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:42.164478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed882447-2b90-4fdf-a08b-a98020e828fd None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.173240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 521/1050] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:42.173941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:42.201770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.202524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.202974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.203239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.203948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:42.204245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.205708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.223670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 530/1051] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:42.224899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:42.237634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:42.282081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7bb80-7a22-4f85-abc2-cce4526ad429 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.316663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.317261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.318056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.318323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.318974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.320355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.337892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852b2ee-6d26-42be-aa3a-34bcefb9f4b4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.342972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 522/1052] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:32:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:42.366397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:42.386806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.387411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.387724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.387993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.388728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] RBAC: Adding query filter params (role.id=21bf7ea27a3a45289ac89c0c64ee9feb, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=4784d64225054ea9a987581a00b6807b) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:42.389035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.390467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.403516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 531/1053] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:42 2022] GET /identity/v3/role_assignments?user.id=4784d64225054ea9a987581a00b6807b&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=21bf7ea27a3a45289ac89c0c64ee9feb => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:42.445577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:42.563967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75563fb-b7dd-4761-8eae-b6cafabf7fc6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.588088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:42.602785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.603423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.603775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.604041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.615438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:42.620171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:42.621104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:42.621968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 523/1054] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:42.627149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:42.735493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757991f-7a17-4cd9-a34d-5a6888e840ab None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.770042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.770738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.771016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.771287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.771975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:42.772276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.773630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.782339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 532/1055] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:42.788204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:42.893916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6f9eb-e84f-4ef6-bbbc-cb47148de262 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:42.928730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:42.929359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:42.929639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:42.929914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] PATH_INFO: `/v3/users/73e50fac58b34ecdabe21128053bfaa6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:42.943252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] RBAC: Authorizing `identity:get_user(user_id=73e50fac58b34ecdabe21128053bfaa6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:42.944738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:42.950921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 524/1056] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:42 2022] GET /identity/v3/users/73e50fac58b34ecdabe21128053bfaa6 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:42.952130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:43.000215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:43.055380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00699668-b348-4b6b-ad83-42481bb84aab None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:43.088704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:43.089312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:43.089590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:43.089860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:43.163325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:43.164789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:43.209269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:43.263548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 533/1057] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:43.269818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:43.318946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:43.378545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdca86-6011-4144-80ec-5cf921aa25f3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:43.412592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a81a0-f0f9-47f0-a410-2752e54428db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:43.413216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a81a0-f0f9-47f0-a410-2752e54428db None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:43.413546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a81a0-f0f9-47f0-a410-2752e54428db None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:43.413770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a81a0-f0f9-47f0-a410-2752e54428db None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:43.414907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a81a0-f0f9-47f0-a410-2752e54428db None admin] RBAC: Authorizing `identity:get_project(project_id=abfea9286b3547d5b51098f0b32047c0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:43.416398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a81a0-f0f9-47f0-a410-2752e54428db None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:43.418227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 525/1058] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:32:43 2022] GET /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:43.424160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:43.521152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839b8f9-bec1-4d7b-a218-1457eb7818bb None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:43.554549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:43.555147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:43.555423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:43.555791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:43.556518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=abfea9286b3547d5b51098f0b32047c0, user.id=73e50fac58b34ecdabe21128053bfaa6) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:43.556826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:43.558239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a4618-e3c6-45dd-aa80-cf7adb55aa7c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:43.563948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 534/1059] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:43 2022] GET /identity/v3/role_assignments?user.id=73e50fac58b34ecdabe21128053bfaa6&scope.project.id=abfea9286b3547d5b51098f0b32047c0&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:44.560219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773325d-6638-4445-a53e-37ebf59dab32 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:44.560735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773325d-6638-4445-a53e-37ebf59dab32 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:44.560735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773325d-6638-4445-a53e-37ebf59dab32 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:44.562258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 526/1060] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:44.567202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:44.567628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:44.567844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:44.585820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:44.586992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:44.590068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:44.591706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5264a-7299-49e1-9ccc-5bfd4c20f182 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:44.592042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5264a-7299-49e1-9ccc-5bfd4c20f182 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:44.592315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5264a-7299-49e1-9ccc-5bfd4c20f182 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:44.593892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 527/1061] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:44.599262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:44.599563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:44.599832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:44.618045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:44.619218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:44.658610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 535/1062] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:44.694983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 528/1063] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:44.925200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:44.984401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.019138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.029144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a24c7-497e-4cc7-85b1-7fd2ff3015cb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.066529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.067667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.068093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.068578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.080066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.083504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:45.090453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:45.091508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:45.092479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 536/1064] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 00:32:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:45.096850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92089845-92cf-4428-a8c6-fd70f5b0657e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.098468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.132039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.132648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.132925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.133450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.203163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.204909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.207729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90264c-894f-40c2-8f9d-82dc875ed6a4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.240724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.244624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.245232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.245504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.245774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.246429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:45.246726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.248113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.262140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 537/1065] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:45.263438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.314037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 529/1066] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 00:32:45.320440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.329086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.371287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96654c-1874-4cad-b08d-0e932c0803a6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.407944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.409012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.409304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.409572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.422565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.424035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.436255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5c32d-99d8-4bb9-b0fb-4487aa92dbdd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:45.437845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:45.438801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:45.439790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:45.439790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:45.439790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:45.439790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 538/1067] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:45.445001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.475748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.476302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.476579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.476846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.479827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.488409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.489863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:45.507249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:45.508633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:45.509562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:45.509562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:45.509562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:45.511892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 530/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:45.514732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.551090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02757626-09cd-46dc-950a-678ea97c7a81 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.565579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.594339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.595106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.595264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.595472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.596308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:45.596616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.598013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.610467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 539/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:45.616516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.641780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a72a-b73a-434e-9b74-5dd9fbf2bd9e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.685237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.686173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.686515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.686828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.687473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:45.687803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.689169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b429f5-2bad-4746-89f1-95f49e4c1221 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.702207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 531/1070] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:45.703420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ccf4a-73cd-4181-ad67-c8973cf687ab None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.703733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ccf4a-73cd-4181-ad67-c8973cf687ab None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.704005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ccf4a-73cd-4181-ad67-c8973cf687ab None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.705247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 532/1071] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:45.710974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.711259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.711578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.728167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b0370-1579-47b1-9757-54521063cf3b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.731788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:45.733037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:45.768743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.769380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.769712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.769919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.778166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.779653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.784799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:45.792459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:45.793499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:45.794471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 540/1072] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:45.801060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:45.819666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 533/1073] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:45.842541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:45.918410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d21be-3e67-4be4-8f89-2aceb3698166 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:45.953993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:45.954595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:45.954870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:45.955138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:45.955911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:45.956213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:45.957627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:45.976159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 541/1074] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:45.982520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.087141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01976811-e29a-4806-8681-aae117ce8a7b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.089122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.119959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.120580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.120852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.121126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.132164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.133865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:46.148414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:46.149461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:46.150360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:46.150360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:46.150360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:46.150360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 534/1075] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:46.155240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.179284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:46.185099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:46.188163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02143b9-725c-4977-8a28-24fcb07bb8d2 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.221893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.222558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.222842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.223114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.232982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:46.237738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:46.238795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:46.239700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:46.244044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 542/1076] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:46.244841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.254117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1be37-dc43-456b-8406-42570df7b668 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.288582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.289181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.289460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.289730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.290367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:46.290668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.292057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.303812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 535/1077] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:46.309304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.344209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e1767-b609-4651-ae1c-2ec333117616 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.377442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.378035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.378315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.378585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.379235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:46.379556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.381466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.389466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 543/1078] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:46.395152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.396388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:46.406249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb0272-2d10-4f44-94df-b3709bf76ecb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.410611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:46.440933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.441552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.441822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.442085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.449202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.450635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:46.461931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:46.462828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:46.463826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 536/1079] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:46.468700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.495125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308556d9-5482-4a32-82cf-d43ad3de9db7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.528227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.528854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.529134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.529406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] PATH_INFO: `/v3/users/73e50fac58b34ecdabe21128053bfaa6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.530537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] RBAC: Authorizing `identity:get_user(user_id=73e50fac58b34ecdabe21128053bfaa6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.532000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.533574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 544/1080] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/users/73e50fac58b34ecdabe21128053bfaa6 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:46.539254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.564692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3886cfc-89c7-471e-b098-d85534e84d4c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.598176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.598821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.599095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.599367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.600085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:46.600392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.601814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.612940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 537/1081] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:46.618623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:46.619309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.631864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:46.638583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042b515-3658-40d0-9b14-ff20045cb77e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.671925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.672554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.672831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.673097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.674207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] RBAC: Authorizing `identity:get_project(project_id=abfea9286b3547d5b51098f0b32047c0)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.675650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.677806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 545/1082] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:32:46 2022] GET /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:46.683541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:46.722905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f27d-3449-4f93-8d3b-60313b093962 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.756556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29271f-6012-4092-b6ec-3b1fcd8597d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.757140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29271f-6012-4092-b6ec-3b1fcd8597d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.757419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29271f-6012-4092-b6ec-3b1fcd8597d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.757692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29271f-6012-4092-b6ec-3b1fcd8597d9 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/4784d64225054ea9a987581a00b6807b/roles/21bf7ea27a3a45289ac89c0c64ee9feb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.760159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29271f-6012-4092-b6ec-3b1fcd8597d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=4784d64225054ea9a987581a00b6807b, role_id=21bf7ea27a3a45289ac89c0c64ee9feb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.761592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29271f-6012-4092-b6ec-3b1fcd8597d9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.774041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 538/1083] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:46 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/4784d64225054ea9a987581a00b6807b/roles/21bf7ea27a3a45289ac89c0c64ee9feb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:32:46.784814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1589-6f87-460d-a676-79c92590f1d5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:46.821781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc2b4b-47f9-453d-8723-0c086820d900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:46.822457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc2b4b-47f9-453d-8723-0c086820d900 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:46.822732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc2b4b-47f9-453d-8723-0c086820d900 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:46.823002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc2b4b-47f9-453d-8723-0c086820d900 None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0/users/73e50fac58b34ecdabe21128053bfaa6/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:46.825042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc2b4b-47f9-453d-8723-0c086820d900 None admin] RBAC: Authorizing `identity:create_grant(project_id=abfea9286b3547d5b51098f0b32047c0, user_id=73e50fac58b34ecdabe21128053bfaa6, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:46.826505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc2b4b-47f9-453d-8723-0c086820d900 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:46.840837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 546/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:46 2022] PUT /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0/users/73e50fac58b34ecdabe21128053bfaa6/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:32:48.909998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6965985-318c-48c6-ba53-b7338e0fd21d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:48.910541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6965985-318c-48c6-ba53-b7338e0fd21d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:48.910630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6965985-318c-48c6-ba53-b7338e0fd21d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:48.912648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 539/1085] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:48.917935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:48.918336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:48.918687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:48.936702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:48.937818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:48.940559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:48.983464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604793c-c98e-4872-bacd-2f97942d009c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:48.983786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604793c-c98e-4872-bacd-2f97942d009c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:48.984061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604793c-c98e-4872-bacd-2f97942d009c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:48.985427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 540/1086] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:32:48.990749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:48.990749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:48.990966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.003339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 547/1087] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:49.009548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:49.010750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:49.070446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:49.076169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 541/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:32:49.276777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.350947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.375074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22675f2c-9749-44a2-af75-2f41e5e07995 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.408195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.408817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.409094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.409361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.416902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:49.420655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:49.425701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:49.427231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:49.428211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 548/1089] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:49.433142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.452317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0902-484e-4d47-a2b4-5a3ce7791be6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.485148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.485871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.486214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.486487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.496714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:32:49.501460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:49.502596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:32:49.503679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:49.507688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 542/1090] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:49.509198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.531929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ad596-cf37-4fa5-8957-998a3a904f12 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.565783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.566382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.566657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.566928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.567644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:49.567946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:49.569303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:49.569922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:49.577201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 549/1091] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:49.583530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.613145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd13fc6-24cc-429e-88a5-bf9a4ddb22b6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.646251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:49.650056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.650661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.650941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.651203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.651893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:49.652189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:49.653547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:49.662180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 543/1092] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:49.667999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.686159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccd31-3132-488a-bd8e-7f927c5561a6 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.719626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.720128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.720409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.720660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.731719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:49.733147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:49.746697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:49.747742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:49.748644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:49.748644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:49.748644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:49.748644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 550/1093] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:49.753430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.771830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dce77-b588-486a-bab6-0061cb8b7e69 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.793847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:49.805863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.806503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.806776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.807041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] PATH_INFO: `/v3/users/73e50fac58b34ecdabe21128053bfaa6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.808164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] RBAC: Authorizing `identity:get_user(user_id=73e50fac58b34ecdabe21128053bfaa6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:49.809582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:49.811277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 544/1094] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/users/73e50fac58b34ecdabe21128053bfaa6 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:49.816696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.851705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac6b57-223e-40f3-9d10-f51f5aae6851 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.864628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:49.884668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.885262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.885540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.885811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:49.886446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:49.886747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:49.888127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:49.899961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 551/1095] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:49.905461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:49.914278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050ef47-6a40-4694-84b0-ba00c6850315 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:49.946928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:49.947544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:49.947819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:49.948082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.000071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91cd65-a309-447a-8d1d-e8521456319d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.002858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.014188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.015658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.033812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.034421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.034697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.034966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.041921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.043672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:32:50.054962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:50.055964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:32:50.056937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 552/1096] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:50.061623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.094688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.113866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 545/1097] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 00:32:50.120020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.159549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767029-54c7-49bf-b515-c6f5ca1aaac1 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.191541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.192132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.192954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.193222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.193941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:50.194236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.195671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.205521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.213375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 553/1098] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:50.216471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9babdb-68ee-4d68-8a8c-daa55756d685 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.219171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.250404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.251020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.251292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.251587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.252677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] RBAC: Authorizing `identity:get_project(project_id=abfea9286b3547d5b51098f0b32047c0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.254092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.256286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 546/1099] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:50.262444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.308258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.316988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03a1ff-c12f-4a7d-8384-a373655c7c6d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.350227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.350822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.351102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.351373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.357444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e649-e69d-4e1d-b65e-e452b78456de None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.392323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.392908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.393182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.393447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.394156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=abfea9286b3547d5b51098f0b32047c0, user.id=73e50fac58b34ecdabe21128053bfaa6) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:50.394457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.395945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.401471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 547/1100] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/role_assignments?user.id=73e50fac58b34ecdabe21128053bfaa6&scope.project.id=abfea9286b3547d5b51098f0b32047c0&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:50.417223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.418793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.514714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.524796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 554/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 00:32:50.530511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.558093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.635634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2d46-2e19-45ac-a7c7-f2d0b1dcd493 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.670853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.671517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.671794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.672056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.683049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.684538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:32:50.698295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:50.699398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:32:50.700330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:32:50.700330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:32:50.700330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:50.700330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 548/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:32:50.705083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.802827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6bfe-56a9-4d76-b883-20969a313e49 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.835801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.836393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.836668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.836934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.837568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:50.837866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.839207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:50.850550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 555/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:50.856213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:50.930453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:50.954923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefa1c-d22f-404c-a712-ec60c7ccb7de None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:50.987806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:50.988423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:50.988699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:50.988979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:50.995361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:50.996821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:32:51.007793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:32:51.008964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:32:51.009899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 549/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:32:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:32:51.015091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:51.066388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:51.112766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25786d6-2772-4f2e-8c11-bbd803264a40 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:51.144736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:51.145823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:51.146098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:51.146364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:51.147047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:51.147347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:51.148808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b093786-b1ec-4d7d-98e8-501654787935 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:51.160498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 556/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:51.167197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:51.263511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf9c9d-1bce-4a99-970c-34f38354a77b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:51.295752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:51.296356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:51.296633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:51.296903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:51.297662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] RBAC: Adding query filter params (role.id=21bf7ea27a3a45289ac89c0c64ee9feb, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=4784d64225054ea9a987581a00b6807b) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:51.297969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:51.299429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e49cc0-a24a-41e2-b3ed-54b410f23116 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:51.301480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:51.307670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 550/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:51 2022] GET /identity/v3/role_assignments?user.id=4784d64225054ea9a987581a00b6807b&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=21bf7ea27a3a45289ac89c0c64ee9feb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:52.575434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1ea35-2cd9-4e34-8c48-93319c43221e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:52.575983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1ea35-2cd9-4e34-8c48-93319c43221e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:52.575983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1ea35-2cd9-4e34-8c48-93319c43221e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:52.577334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 557/1107] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:52.582405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:52.582795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:52.582970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:52.600645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:52.601816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:52.665904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 551/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:52.935665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1ea35-2cd9-4e34-8c48-93319c43221e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:53.030380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1ea35-2cd9-4e34-8c48-93319c43221e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:53.043228 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:53.063570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f101b-b65d-4363-8f0e-51a71e55a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:53.064171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f101b-b65d-4363-8f0e-51a71e55a082 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:53.064446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f101b-b65d-4363-8f0e-51a71e55a082 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:53.064713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f101b-b65d-4363-8f0e-51a71e55a082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:53.065789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f101b-b65d-4363-8f0e-51a71e55a082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:53.067243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f101b-b65d-4363-8f0e-51a71e55a082 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:53.068799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 558/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:53.074399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:53.169103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7693-b9ef-4deb-8708-e7abc2091cce None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:53.195197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:53.202947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be26f83-ad56-4611-bb84-c78c74bcced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:53.203589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be26f83-ad56-4611-bb84-c78c74bcced3 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:53.203867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be26f83-ad56-4611-bb84-c78c74bcced3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:53.204136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be26f83-ad56-4611-bb84-c78c74bcced3 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:53.204810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be26f83-ad56-4611-bb84-c78c74bcced3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:53.206230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be26f83-ad56-4611-bb84-c78c74bcced3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:53.228124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 552/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:32:53 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:32:55.391787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56060837-aef4-41ad-9bcf-34a0acd7c2c0 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:55.392338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56060837-aef4-41ad-9bcf-34a0acd7c2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:55.392338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56060837-aef4-41ad-9bcf-34a0acd7c2c0 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:55.393690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 559/1111] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:55.399370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:55.399793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:55.399940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:55.418601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:55.419790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:55.482537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 553/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:55.750800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56060837-aef4-41ad-9bcf-34a0acd7c2c0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:55.849583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56060837-aef4-41ad-9bcf-34a0acd7c2c0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:55.881799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:55.882404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:55.882681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:55.883193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] PATH_INFO: `/v3/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:55.884596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] RBAC: Authorizing `identity:get_role(role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:55.885942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:55.887236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 560/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:55 2022] GET /identity/v3/roles/1d2b7e763c90438eab518a55d7589260 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:55.892825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:55.985896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63085977-4e3b-4f9d-9549-25d7ccf2c379 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:55.998826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:56.019709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:56.020903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:56.021177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:56.021445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] PATH_INFO: `/v3/users/ebb31b35df944f56910caca54f64e826` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:56.033730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] RBAC: Authorizing `identity:get_user(user_id=ebb31b35df944f56910caca54f64e826)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:56.035155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:56.036760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 554/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:55 2022] GET /identity/v3/users/ebb31b35df944f56910caca54f64e826 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:56.042351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:56.089242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:56.137865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc256bb-4af6-449f-ab49-4e0932d230c9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:56.169283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:56.169857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:56.170129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:56.170392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:56.233758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:56.235170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:56.279921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:56.328892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 561/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:32:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:32:56.334763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:56.429471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462a601-4710-4f77-b6c8-799f54929777 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:56.461691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a84f8-269a-4936-b961-8137d2cd8479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:56.462299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a84f8-269a-4936-b961-8137d2cd8479 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:56.462575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a84f8-269a-4936-b961-8137d2cd8479 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:56.462846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a84f8-269a-4936-b961-8137d2cd8479 None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:56.463972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a84f8-269a-4936-b961-8137d2cd8479 None admin] RBAC: Authorizing `identity:get_project(project_id=abfea9286b3547d5b51098f0b32047c0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:56.465386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a84f8-269a-4936-b961-8137d2cd8479 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:56.466960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 555/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:32:56 2022] GET /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:56.472889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:56.568000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fcb34-4ba0-401b-856d-a05a1b344a58 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:56.599757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:56.600337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:56.600607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:56.600872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:56.601579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] RBAC: Adding query filter params (role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=abfea9286b3547d5b51098f0b32047c0, user.id=ebb31b35df944f56910caca54f64e826) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:32:56.601880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:56.603273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077eef-e771-4a1f-91ab-eb2ba446be1d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:56.608379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 562/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:32:56 2022] GET /identity/v3/role_assignments?user.id=ebb31b35df944f56910caca54f64e826&scope.project.id=abfea9286b3547d5b51098f0b32047c0&role.id=1d2b7e763c90438eab518a55d7589260 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:58.715921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ffd6-fce4-4565-9310-81ad9972d45c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:58.716382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ffd6-fce4-4565-9310-81ad9972d45c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:58.716493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ffd6-fce4-4565-9310-81ad9972d45c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:58.717807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 556/1118] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:32:58.722995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:58.723963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:58.724192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:58.741962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:32:58.743149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:32:58.750394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:58.807879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 563/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:32:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:32:59.076067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ffd6-fce4-4565-9310-81ad9972d45c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:59.117808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:59.173545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ffd6-fce4-4565-9310-81ad9972d45c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:59.205990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:59.206590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:59.206866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:59.207136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] PATH_INFO: `/v3/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:59.208545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] RBAC: Authorizing `identity:get_role(role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:59.209897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:59.211166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 557/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:59 2022] GET /identity/v3/roles/1d2b7e763c90438eab518a55d7589260 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:59.216829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:59.310513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707234a-594e-4771-84d9-476055e4e4e4 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:59.343179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:59.343753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:59.344024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:59.344287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] PATH_INFO: `/v3/users/ebb31b35df944f56910caca54f64e826` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:59.345375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] RBAC: Authorizing `identity:get_user(user_id=ebb31b35df944f56910caca54f64e826)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:59.346776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:59.348316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 564/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:32:59 2022] GET /identity/v3/users/ebb31b35df944f56910caca54f64e826 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:32:59.353711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:59.449164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa34599-43f7-47ad-a0ab-412b2518578d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:59.457887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:59.482000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20431a0-39f3-4306-97cf-2097a24018aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:59.482600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20431a0-39f3-4306-97cf-2097a24018aa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:59.482874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20431a0-39f3-4306-97cf-2097a24018aa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:59.483141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20431a0-39f3-4306-97cf-2097a24018aa None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:59.484263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20431a0-39f3-4306-97cf-2097a24018aa None admin] RBAC: Authorizing `identity:get_project(project_id=abfea9286b3547d5b51098f0b32047c0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:59.485671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20431a0-39f3-4306-97cf-2097a24018aa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:59.487240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 558/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:32:59 2022] GET /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:32:59.493500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:32:59.505281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:32:59.587051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2f3d-5d10-43c6-9220-bd44b78d971e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:32:59.619473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a53ab3-f779-427b-b679-e62d29c7f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:32:59.620078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53ab3-f779-427b-b679-e62d29c7f7d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:32:59.620351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53ab3-f779-427b-b679-e62d29c7f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:32:59.620615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53ab3-f779-427b-b679-e62d29c7f7d1 None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0/users/ebb31b35df944f56910caca54f64e826/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:32:59.622697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a53ab3-f779-427b-b679-e62d29c7f7d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=abfea9286b3547d5b51098f0b32047c0, user_id=ebb31b35df944f56910caca54f64e826, role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:32:59.624135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a53ab3-f779-427b-b679-e62d29c7f7d1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:32:59.636031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 565/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:32:59 2022] PUT /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0/users/ebb31b35df944f56910caca54f64e826/roles/1d2b7e763c90438eab518a55d7589260 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:33:01.750000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046f63-24bf-4a9f-8a20-83cc5e8ce9e8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:01.750454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046f63-24bf-4a9f-8a20-83cc5e8ce9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:01.750541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046f63-24bf-4a9f-8a20-83cc5e8ce9e8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:01.751895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 559/1124] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:01.757540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:01.758168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:01.758340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:01.776519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:01.777681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:01.837115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:01.842405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 566/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:02.103726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3046f63-24bf-4a9f-8a20-83cc5e8ce9e8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:02.200961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3046f63-24bf-4a9f-8a20-83cc5e8ce9e8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:02.233515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:02.234127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:02.234404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:02.234670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] PATH_INFO: `/v3/roles/1d2b7e763c90438eab518a55d7589260` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:02.236081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] RBAC: Authorizing `identity:get_role(role_id=1d2b7e763c90438eab518a55d7589260)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:02.237441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:02.239264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 560/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:02 2022] GET /identity/v3/roles/1d2b7e763c90438eab518a55d7589260 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:02.244835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:02.337916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f93598-68e8-49ba-ab9d-cc18cc92e8d3 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:02.369946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:02.370494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:02.370770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:02.371039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] PATH_INFO: `/v3/users/ebb31b35df944f56910caca54f64e826` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:02.372162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] RBAC: Authorizing `identity:get_user(user_id=ebb31b35df944f56910caca54f64e826)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:02.373578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:02.375103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 567/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:02 2022] GET /identity/v3/users/ebb31b35df944f56910caca54f64e826 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:02.381181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:02.424571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:02.477273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce28a75-6913-44f0-aad8-fd493cb87312 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:02.514687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:02.515283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:02.515586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:02.515857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:02.579718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:02.581180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:02.623061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:02.675796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 561/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:33:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:33:02.682184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:02.745437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:02.776706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ee96-6230-4ba3-bcd2-72ef46a5e464 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:02.810304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf6da4-a810-48cc-999f-93df07c7da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:02.810925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf6da4-a810-48cc-999f-93df07c7da87 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:02.811202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf6da4-a810-48cc-999f-93df07c7da87 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:02.811471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf6da4-a810-48cc-999f-93df07c7da87 None admin] PATH_INFO: `/v3/projects/abfea9286b3547d5b51098f0b32047c0` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:02.812603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf6da4-a810-48cc-999f-93df07c7da87 None admin] RBAC: Authorizing `identity:get_project(project_id=abfea9286b3547d5b51098f0b32047c0)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:02.814019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf6da4-a810-48cc-999f-93df07c7da87 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:02.815677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 568/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:02 2022] GET /identity/v3/projects/abfea9286b3547d5b51098f0b32047c0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:02.821852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:02.915776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28699ed-4e23-431c-8393-ede12e5b8728 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:02.947645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:02.948236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:02.948513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:02.948781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:02.949496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] RBAC: Adding query filter params (role.id=1d2b7e763c90438eab518a55d7589260, scope.project.id=abfea9286b3547d5b51098f0b32047c0, user.id=ebb31b35df944f56910caca54f64e826) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:02.949801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:02.951237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fbbab-8a47-4878-8b76-3937ca731e26 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:02.956730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 562/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:33:02 2022] GET /identity/v3/role_assignments?user.id=ebb31b35df944f56910caca54f64e826&scope.project.id=abfea9286b3547d5b51098f0b32047c0&role.id=1d2b7e763c90438eab518a55d7589260 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:05.170079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ccd42-a5ef-49a3-a42f-f359c8814ebe None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:05.170669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ccd42-a5ef-49a3-a42f-f359c8814ebe None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:05.170669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ccd42-a5ef-49a3-a42f-f359c8814ebe None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:05.171980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 569/1131] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:05.177742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:05.178150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:05.178288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:05.201738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:05.202929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:05.204884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:05.268383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 563/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:05.539101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ccd42-a5ef-49a3-a42f-f359c8814ebe None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:05.635244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ccd42-a5ef-49a3-a42f-f359c8814ebe None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:05.667332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26e18-4c69-4643-8150-e529f3a80e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:05.667970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26e18-4c69-4643-8150-e529f3a80e16 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:05.668245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26e18-4c69-4643-8150-e529f3a80e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:05.668510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26e18-4c69-4643-8150-e529f3a80e16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:05.669657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26e18-4c69-4643-8150-e529f3a80e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:05.671069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26e18-4c69-4643-8150-e529f3a80e16 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:05.672640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 570/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:05.678364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:05.780981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b4b7c-1c87-4292-8636-7130fcf9fef3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:05.812983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f9e67-0f61-44d0-a6ee-06dabdea2b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:05.813519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9e67-0f61-44d0-a6ee-06dabdea2b5b None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:05.813797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9e67-0f61-44d0-a6ee-06dabdea2b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:05.814067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9e67-0f61-44d0-a6ee-06dabdea2b5b None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:05.814705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f9e67-0f61-44d0-a6ee-06dabdea2b5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:05.816169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f9e67-0f61-44d0-a6ee-06dabdea2b5b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:05.825742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:05.844794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 564/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:33:05 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:33:08.019368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940fcd5-12ff-4f88-aa98-a11df0253d60 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:08.020431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940fcd5-12ff-4f88-aa98-a11df0253d60 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:08.020540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940fcd5-12ff-4f88-aa98-a11df0253d60 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:08.021809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 571/1135] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:08.027201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:08.027651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:08.027783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:08.045287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:08.046459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:08.109807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 565/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:08.381148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940fcd5-12ff-4f88-aa98-a11df0253d60 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:08.385207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:08.478392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940fcd5-12ff-4f88-aa98-a11df0253d60 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:08.512184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12181bbe-39a9-41af-84fe-faa763cd1ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:08.512800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181bbe-39a9-41af-84fe-faa763cd1ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:08.513070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181bbe-39a9-41af-84fe-faa763cd1ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:08.513331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181bbe-39a9-41af-84fe-faa763cd1ff3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:08.514411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12181bbe-39a9-41af-84fe-faa763cd1ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:08.515923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12181bbe-39a9-41af-84fe-faa763cd1ff3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:08.517460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 572/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:33:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:08.523267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:08.617804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0f8e7-81e8-403a-8caf-ee2d7209cf9d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:08.636834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:08.650897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787993f-6daf-42f2-afa2-eb005a674ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:08.651520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787993f-6daf-42f2-afa2-eb005a674ed9 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:08.651807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787993f-6daf-42f2-afa2-eb005a674ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:08.652078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787993f-6daf-42f2-afa2-eb005a674ed9 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:08.652717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787993f-6daf-42f2-afa2-eb005a674ed9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:08.654138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787993f-6daf-42f2-afa2-eb005a674ed9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:08.674510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 566/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:33:08 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:33:10.826057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de58c8-f021-4e01-9424-25bbc13461e9 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:10.826562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de58c8-f021-4e01-9424-25bbc13461e9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:10.826648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de58c8-f021-4e01-9424-25bbc13461e9 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:10.827953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 573/1139] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:10.833737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:10.834149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:10.834261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:10.851920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:10.853086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:10.917065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 567/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:11.182943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de58c8-f021-4e01-9424-25bbc13461e9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:11.249738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:11.278749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de58c8-f021-4e01-9424-25bbc13461e9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:11.310738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:11.311341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:11.311640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:11.311908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:11.321988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:33:11.326694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:11.327797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:33:11.328834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 574/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:33:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:11.333848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:11.427878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38ce24-a7aa-41b9-8e3a-ae6d2bf1cdd2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:11.434849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:11.461876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:11.462467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:11.462743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:11.463011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:11.463697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:11.463999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:11.465356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:11.473546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 568/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:33:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:11.479312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:11.574032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c86fd-00ba-461a-93cf-6929e62b06f7 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:11.601937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:11.606876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:11.607499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:11.607780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:11.608048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] PATH_INFO: `/v3/users/55960fe8e2ef4a1aa5f38cee30faf126` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:11.621200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] RBAC: Authorizing `identity:get_user(user_id=55960fe8e2ef4a1aa5f38cee30faf126)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:11.622606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:11.624195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 575/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:11 2022] GET /identity/v3/users/55960fe8e2ef4a1aa5f38cee30faf126 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:11.629719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:11.726122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d34a0b-e036-487f-811d-998540530a5f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:11.759656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:11.760250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:11.760526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:11.760793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:11.793138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:11.824914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:11.826353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:11.920579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 569/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:33:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:33:11.926509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:12.022832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74081-460d-40bc-8e91-d9318b9447e4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:12.054678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fce20-8a4e-4742-9e7d-9c841d7ae195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:12.055286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fce20-8a4e-4742-9e7d-9c841d7ae195 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:12.055583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fce20-8a4e-4742-9e7d-9c841d7ae195 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:12.055847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fce20-8a4e-4742-9e7d-9c841d7ae195 None admin] PATH_INFO: `/v3/projects/3e4bd239fb5e453a923d279e5ba60e97` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:12.056929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fce20-8a4e-4742-9e7d-9c841d7ae195 None admin] RBAC: Authorizing `identity:get_project(project_id=3e4bd239fb5e453a923d279e5ba60e97)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:12.058331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fce20-8a4e-4742-9e7d-9c841d7ae195 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:12.059946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 576/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:11 2022] GET /identity/v3/projects/3e4bd239fb5e453a923d279e5ba60e97 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:12.065920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:12.129629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:12.161929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab600b8b-a442-488d-b7bf-ad6b4543c09c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:12.193475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:12.194057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:12.194327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:12.194589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:12.195294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=3e4bd239fb5e453a923d279e5ba60e97, user.id=55960fe8e2ef4a1aa5f38cee30faf126) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:12.195624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:12.197029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a407ffd-5c59-45bc-9278-804c73f4cc4b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:12.202096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 570/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:33:12 2022] GET /identity/v3/role_assignments?user.id=55960fe8e2ef4a1aa5f38cee30faf126&scope.project.id=3e4bd239fb5e453a923d279e5ba60e97&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:14.310485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db633d-bf0e-4077-8c90-1541544923da None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:14.311110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db633d-bf0e-4077-8c90-1541544923da None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:14.311110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db633d-bf0e-4077-8c90-1541544923da None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:14.312395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 577/1147] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:14.318121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:14.318558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:14.318659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:14.336406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:14.337583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:14.402091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 571/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 00:33:14.668644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db633d-bf0e-4077-8c90-1541544923da None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:14.725648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:14.766035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db633d-bf0e-4077-8c90-1541544923da None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:14.798190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:14.798858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:14.799138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:14.799410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:14.810140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:33:14.814960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:14.816140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:33:14.817068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 578/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:14.822050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:14.879867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:14.917595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63a84a-ad49-46c4-8243-44c2f7eecbfe None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:14.949189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:14.949715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:14.949986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:14.950250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:14.950896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:14.951192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:14.952574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:14.960728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 572/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:14.966486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:15.063372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563db6b8-98d6-46ea-9d3a-331f99ff913f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:15.081479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:15.098838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:15.099456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:15.099756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:15.100024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] PATH_INFO: `/v3/users/55960fe8e2ef4a1aa5f38cee30faf126` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:15.101133 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] RBAC: Authorizing `identity:get_user(user_id=55960fe8e2ef4a1aa5f38cee30faf126)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:15.102566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:15.104186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 579/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:14 2022] GET /identity/v3/users/55960fe8e2ef4a1aa5f38cee30faf126 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:15.110037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:15.210067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60849f05-d6b7-4b21-a143-da453bb23b12 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:15.232156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:15.243223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd384c-8e0b-4eeb-be34-a6c16acce840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:15.243866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd384c-8e0b-4eeb-be34-a6c16acce840 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:15.244139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd384c-8e0b-4eeb-be34-a6c16acce840 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:15.244403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd384c-8e0b-4eeb-be34-a6c16acce840 None admin] PATH_INFO: `/v3/projects/3e4bd239fb5e453a923d279e5ba60e97` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:15.245486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd384c-8e0b-4eeb-be34-a6c16acce840 None admin] RBAC: Authorizing `identity:get_project(project_id=3e4bd239fb5e453a923d279e5ba60e97)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:15.246904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd384c-8e0b-4eeb-be34-a6c16acce840 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:15.248517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 573/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:15 2022] GET /identity/v3/projects/3e4bd239fb5e453a923d279e5ba60e97 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:15.255056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:15.349812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1205-83fd-4227-b456-19b3a8f20e2a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:15.382253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c26df9-bc93-4683-88a7-98d94f3bdb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:15.383146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c26df9-bc93-4683-88a7-98d94f3bdb5b None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:15.383420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c26df9-bc93-4683-88a7-98d94f3bdb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:15.383710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c26df9-bc93-4683-88a7-98d94f3bdb5b None admin] PATH_INFO: `/v3/projects/3e4bd239fb5e453a923d279e5ba60e97/users/55960fe8e2ef4a1aa5f38cee30faf126/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:15.385589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c26df9-bc93-4683-88a7-98d94f3bdb5b None admin] RBAC: Authorizing `identity:create_grant(project_id=3e4bd239fb5e453a923d279e5ba60e97, user_id=55960fe8e2ef4a1aa5f38cee30faf126, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:15.386985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c26df9-bc93-4683-88a7-98d94f3bdb5b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:15.400231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 580/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:33:15 2022] PUT /identity/v3/projects/3e4bd239fb5e453a923d279e5ba60e97/users/55960fe8e2ef4a1aa5f38cee30faf126/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:33:17.506337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd068d5-584a-4b9e-ba5c-a2964ed0d011 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:17.506894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd068d5-584a-4b9e-ba5c-a2964ed0d011 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:17.506894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd068d5-584a-4b9e-ba5c-a2964ed0d011 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:17.508605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 574/1154] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:17.513725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:17.514163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:17.514268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:17.531964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:17.533139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:17.574033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:17.597372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 581/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:17.864492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd068d5-584a-4b9e-ba5c-a2964ed0d011 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:17.962004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd068d5-584a-4b9e-ba5c-a2964ed0d011 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:17.993713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:17.994319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:17.994594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:17.994864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:18.005616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:33:18.010596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:18.011744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:33:18.012625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 575/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:18.017528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:18.115300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df498439-607d-4fee-8ba4-520144c4f546 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:18.147397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:18.148077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:18.148351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:18.148624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:18.149272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:18.149569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:18.150912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:18.158487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 582/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:18.165088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:18.215553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:18.260748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79ce1f-4eee-4f88-ad43-8ec5945937d1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:18.293033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:18.293639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:18.293910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:18.294177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] PATH_INFO: `/v3/users/55960fe8e2ef4a1aa5f38cee30faf126` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:18.295262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] RBAC: Authorizing `identity:get_user(user_id=55960fe8e2ef4a1aa5f38cee30faf126)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:18.296708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:18.298236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 576/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:18 2022] GET /identity/v3/users/55960fe8e2ef4a1aa5f38cee30faf126 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:18.303942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:18.342633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:18.399706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51126-383c-455d-89fb-d8922c4f6c80 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:18.430955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:18.431556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:18.431830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:18.432094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:18.494328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:18.495764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:18.532599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:18.589187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 583/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:33:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:33:18.595097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:18.693619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82b3e5-f494-4d9f-8d0c-a7784793589b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:18.712107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:18.725657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06128eb5-99d5-4fdf-a859-4380c0a2911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:18.726275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06128eb5-99d5-4fdf-a859-4380c0a2911d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:18.726551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06128eb5-99d5-4fdf-a859-4380c0a2911d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:18.726822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06128eb5-99d5-4fdf-a859-4380c0a2911d None admin] PATH_INFO: `/v3/projects/3e4bd239fb5e453a923d279e5ba60e97` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:18.727952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06128eb5-99d5-4fdf-a859-4380c0a2911d None admin] RBAC: Authorizing `identity:get_project(project_id=3e4bd239fb5e453a923d279e5ba60e97)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:18.729375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06128eb5-99d5-4fdf-a859-4380c0a2911d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:18.730987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 577/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:18 2022] GET /identity/v3/projects/3e4bd239fb5e453a923d279e5ba60e97 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:18.737090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:18.831070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918730-bac6-40c4-af7b-c180a618896d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:18.862515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:18.863094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:18.863368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:18.864169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:18.864882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=3e4bd239fb5e453a923d279e5ba60e97, user.id=55960fe8e2ef4a1aa5f38cee30faf126) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:18.865184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:18.866587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f153b-6f58-456d-a3ba-d3c1c6598a9e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:18.872098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 584/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:33:18 2022] GET /identity/v3/role_assignments?user.id=55960fe8e2ef4a1aa5f38cee30faf126&scope.project.id=3e4bd239fb5e453a923d279e5ba60e97&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:21.087967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90432f4-3b0b-4999-a31c-f055847c0489 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:21.088500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90432f4-3b0b-4999-a31c-f055847c0489 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:21.088500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90432f4-3b0b-4999-a31c-f055847c0489 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:21.089820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 578/1162] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:21.094945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:21.095373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:21.095499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:21.102275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:21.114745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:21.115955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:21.180626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 585/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:21.451239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90432f4-3b0b-4999-a31c-f055847c0489 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:21.551841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90432f4-3b0b-4999-a31c-f055847c0489 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:21.583595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:21.584184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:21.584464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:21.584732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:21.636841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:21.649644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:21.651091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:21.744467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 579/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:33:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:33:21.750540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:21.846092 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a63dd6-17e3-4e8d-a58e-89b4af0d6e6b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:21.874032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:21.878925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:21.880134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:21.880408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:21.880673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:21.887323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:21.888767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-dd63d621-7fdd-4993-8275-e470b617f4b1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:21.899728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:21.900983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:33:21.902122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 586/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:33:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:21.906794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:21.989929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:22.001584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16de91-27a0-4f12-84fa-f1f0685565be None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:22.033276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:22.033859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:22.034133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:22.034401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:22.035027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:22.035862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:22.037205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39067b72-28df-4422-8276-70dd6683f674 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:22.043980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 580/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 00:33:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:24.162044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde4866-f07a-4778-bc99-35c878f28aea None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:24.162601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde4866-f07a-4778-bc99-35c878f28aea None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:24.162601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde4866-f07a-4778-bc99-35c878f28aea None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:24.163948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 587/1167] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:24.169277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d21c7-aec9-4985-a91b-0fad60094599 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:24.169743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d21c7-aec9-4985-a91b-0fad60094599 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:24.169847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d21c7-aec9-4985-a91b-0fad60094599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:24.188745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9d4d21c7-aec9-4985-a91b-0fad60094599 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:24.189933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9d4d21c7-aec9-4985-a91b-0fad60094599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:24.254067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 581/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:24.521627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde4866-f07a-4778-bc99-35c878f28aea None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:24.618738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde4866-f07a-4778-bc99-35c878f28aea None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:24.651126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750e891-7350-4942-be21-328afd34e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:24.651753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750e891-7350-4942-be21-328afd34e085 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:24.652032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750e891-7350-4942-be21-328afd34e085 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:24.652302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750e891-7350-4942-be21-328afd34e085 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:24.652973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750e891-7350-4942-be21-328afd34e085 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:24.654321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750e891-7350-4942-be21-328afd34e085 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:24.669823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 588/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Thu Sep 15 00:33:24 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:33:26.827638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e59efc-a87c-4245-a4b5-9295ab1a2df4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:26.828147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e59efc-a87c-4245-a4b5-9295ab1a2df4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:26.828147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e59efc-a87c-4245-a4b5-9295ab1a2df4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:26.829494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 582/1170] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:26.835295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:26.835746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:26.835850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:26.854904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:26.856107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:26.900667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:26.920777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 589/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:27.190154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e59efc-a87c-4245-a4b5-9295ab1a2df4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:27.269093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:27.288417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e59efc-a87c-4245-a4b5-9295ab1a2df4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:27.320798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9e83-abaa-4f9b-9279-5073b2e2f18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:27.321401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9e83-abaa-4f9b-9279-5073b2e2f18f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:27.321671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9e83-abaa-4f9b-9279-5073b2e2f18f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:27.321932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9e83-abaa-4f9b-9279-5073b2e2f18f None admin] PATH_INFO: `/v3/domains/5fb45a11ed1e4d7d8e1fe30bfc820642` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:27.332483 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9e83-abaa-4f9b-9279-5073b2e2f18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=5fb45a11ed1e4d7d8e1fe30bfc820642)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:27.333914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9e83-abaa-4f9b-9279-5073b2e2f18f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:27.335452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 583/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Thu Sep 15 00:33:27 2022] GET /identity/v3/domains/5fb45a11ed1e4d7d8e1fe30bfc820642 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:27.341181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:27.435677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129e9a-2cc9-4ca1-8a07-2f035e2ea36d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:27.469038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548facb-7c13-4a2a-b643-ee34c22e7aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:27.469561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548facb-7c13-4a2a-b643-ee34c22e7aa0 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:27.469835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548facb-7c13-4a2a-b643-ee34c22e7aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:27.470098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548facb-7c13-4a2a-b643-ee34c22e7aa0 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:27.470745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548facb-7c13-4a2a-b643-ee34c22e7aa0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:27.472176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548facb-7c13-4a2a-b643-ee34c22e7aa0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:27.496672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 590/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:33:27 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:33:29.625464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7910c2-6f64-4c16-b249-547d9c197da5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:29.626014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7910c2-6f64-4c16-b249-547d9c197da5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:29.626014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7910c2-6f64-4c16-b249-547d9c197da5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:29.627322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 584/1174] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:29.632485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:29.633116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:29.633283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:29.650956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:29.652156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:29.679956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:29.718847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 591/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:29.992042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7910c2-6f64-4c16-b249-547d9c197da5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:30.090627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7910c2-6f64-4c16-b249-547d9c197da5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:30.122358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b274b-aae5-4659-b69c-856b6faf2f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:30.122972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b274b-aae5-4659-b69c-856b6faf2f40 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:30.123245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b274b-aae5-4659-b69c-856b6faf2f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:30.123535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b274b-aae5-4659-b69c-856b6faf2f40 None admin] PATH_INFO: `/v3/domains/5fb45a11ed1e4d7d8e1fe30bfc820642` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:30.124627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b274b-aae5-4659-b69c-856b6faf2f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5fb45a11ed1e4d7d8e1fe30bfc820642)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:30.126046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b274b-aae5-4659-b69c-856b6faf2f40 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:30.127596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 585/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Thu Sep 15 00:33:29 2022] GET /identity/v3/domains/5fb45a11ed1e4d7d8e1fe30bfc820642 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:30.133656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:30.226715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe9a0-6893-43ce-a036-cb6efa4420c4 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:30.259364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cdf27-dd9e-44b0-98a7-fb39e1a232bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:30.259918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cdf27-dd9e-44b0-98a7-fb39e1a232bb None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:30.260190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cdf27-dd9e-44b0-98a7-fb39e1a232bb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:30.260455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cdf27-dd9e-44b0-98a7-fb39e1a232bb None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:30.261096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cdf27-dd9e-44b0-98a7-fb39e1a232bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:30.262497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cdf27-dd9e-44b0-98a7-fb39e1a232bb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:30.283948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 592/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:33:30 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:33:32.442653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847717c-2724-4230-ac57-feccc48674f4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:32.443194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847717c-2724-4230-ac57-feccc48674f4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:32.443194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847717c-2724-4230-ac57-feccc48674f4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:32.444560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 586/1178] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:32.449844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:32.450212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:32.450397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:32.468202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:32.469388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:32.483346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:32.535473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 593/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:32.805588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847717c-2724-4230-ac57-feccc48674f4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:32.818702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:32.905884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847717c-2724-4230-ac57-feccc48674f4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:32.937858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:32.938466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:32.938745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:32.939016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:32.949812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:33:32.954610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:32.955732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:33:32.956680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 587/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:32.961591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:33.055074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32333c-3f48-49b3-b2b7-c37cc83fee10 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:33.086981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:33.087589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:33.087864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:33.088130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:33.088782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:33.089078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:33.090425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:33.097963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 594/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:33:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:33.103608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:33.169237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:33.200377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae7be-2d6f-4487-acd9-c4f878682eee None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:33.233737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:33.234349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:33.234621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:33.234886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] PATH_INFO: `/v3/users/b0b0ab4e8f1e4c44b239f014be471305` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:33.247908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b0ab4e8f1e4c44b239f014be471305)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:33.249338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:33.250904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 588/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:33 2022] GET /identity/v3/users/b0b0ab4e8f1e4c44b239f014be471305 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:33.256703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:33.272412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:33.353291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856dfde-6459-4c2f-9af4-210a33ddf987 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:33.384750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:33.385335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:33.385608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:33.385871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:33.448128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:33.449553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:33.462098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:33.545011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 595/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:33:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:33:33.551566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:33.646852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23783782-aa81-4bf6-a2e5-67bd312b3698 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:33.672256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:33.680779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abb58b-bdb9-45cf-820f-65e8d71da70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:33.681390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abb58b-bdb9-45cf-820f-65e8d71da70c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:33.681662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abb58b-bdb9-45cf-820f-65e8d71da70c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:33.681928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abb58b-bdb9-45cf-820f-65e8d71da70c None admin] PATH_INFO: `/v3/projects/c3bac4a7fc284b9db681d1b007904f3a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:33.683019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abb58b-bdb9-45cf-820f-65e8d71da70c None admin] RBAC: Authorizing `identity:get_project(project_id=c3bac4a7fc284b9db681d1b007904f3a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:33.684471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abb58b-bdb9-45cf-820f-65e8d71da70c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:33.686059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 589/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:33 2022] GET /identity/v3/projects/c3bac4a7fc284b9db681d1b007904f3a => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:33.692214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:33.786310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141c2b4-a7b8-4c52-8fa6-e117f9dfd75f None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:33.798695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:33.818778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:33.819371 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:33.819672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:33.819932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:33.820646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=c3bac4a7fc284b9db681d1b007904f3a, user.id=b0b0ab4e8f1e4c44b239f014be471305) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:33.820953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:33.822348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912a47-bb44-4fa0-a50d-04a2548eb852 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:33.827862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 596/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:33:33 2022] GET /identity/v3/role_assignments?user.id=b0b0ab4e8f1e4c44b239f014be471305&scope.project.id=c3bac4a7fc284b9db681d1b007904f3a&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:35.929089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ed435-3130-4f39-9b77-e58431d576d9 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:35.929613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ed435-3130-4f39-9b77-e58431d576d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:35.929613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ed435-3130-4f39-9b77-e58431d576d9 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:35.930869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 590/1186] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:35.935958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:35.936370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:35.936506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:35.954083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:35.955259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:36.019933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 597/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:36.287253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ed435-3130-4f39-9b77-e58431d576d9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:36.384752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ed435-3130-4f39-9b77-e58431d576d9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:36.417004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:36.417610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:36.417882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:36.418146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:36.428020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:33:36.432750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:36.434544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:33:36.436215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 591/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:33:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:36.442109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:36.537766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1fa72-e9cd-4d7a-bf65-528b14be78ee None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:36.544618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:36.570322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:36.570913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:36.571182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:36.571444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:36.572120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:36.572416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:36.573761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:36.581931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 598/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:33:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:36.588004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:36.642461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:36.685124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840b330-d5c8-454f-b79d-5b646d80b6e5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:36.717833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:36.718450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:36.718730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:36.718998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] PATH_INFO: `/v3/users/b0b0ab4e8f1e4c44b239f014be471305` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:36.720121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] RBAC: Authorizing `identity:get_user(user_id=b0b0ab4e8f1e4c44b239f014be471305)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:36.721545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:36.723178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 592/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:36 2022] GET /identity/v3/users/b0b0ab4e8f1e4c44b239f014be471305 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:36.728969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:36.827027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aa6a9-68b1-4f1a-848d-d4354958cb2d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:36.857882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523f933-338b-48cc-bb2e-052502831afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:36.858492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523f933-338b-48cc-bb2e-052502831afb None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:36.858765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523f933-338b-48cc-bb2e-052502831afb None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:36.859027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523f933-338b-48cc-bb2e-052502831afb None admin] PATH_INFO: `/v3/projects/c3bac4a7fc284b9db681d1b007904f3a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:36.860122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523f933-338b-48cc-bb2e-052502831afb None admin] RBAC: Authorizing `identity:get_project(project_id=c3bac4a7fc284b9db681d1b007904f3a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:36.861525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523f933-338b-48cc-bb2e-052502831afb None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:36.863659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 599/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:36 2022] GET /identity/v3/projects/c3bac4a7fc284b9db681d1b007904f3a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:36.869602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:36.963195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014272dc-ce06-401e-9611-0a13c25fe56b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:36.985373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:36.996570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ce45e-298a-4098-8fff-ab6a994f1b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:36.997226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ce45e-298a-4098-8fff-ab6a994f1b2b None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:36.997502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ce45e-298a-4098-8fff-ab6a994f1b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:36.997772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ce45e-298a-4098-8fff-ab6a994f1b2b None admin] PATH_INFO: `/v3/projects/c3bac4a7fc284b9db681d1b007904f3a/users/b0b0ab4e8f1e4c44b239f014be471305/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:36.999741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ce45e-298a-4098-8fff-ab6a994f1b2b None admin] RBAC: Authorizing `identity:create_grant(project_id=c3bac4a7fc284b9db681d1b007904f3a, user_id=b0b0ab4e8f1e4c44b239f014be471305, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:37.001172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ce45e-298a-4098-8fff-ab6a994f1b2b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:37.013292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 593/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:33:36 2022] PUT /identity/v3/projects/c3bac4a7fc284b9db681d1b007904f3a/users/b0b0ab4e8f1e4c44b239f014be471305/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:33:39.108316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b762-68a2-45b4-83c7-fbb24078966c None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:39.108837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b762-68a2-45b4-83c7-fbb24078966c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:39.108837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b762-68a2-45b4-83c7-fbb24078966c None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:39.110177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 600/1193] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:33:39.115329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:39.115753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:39.115912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:39.133966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:33:39.135139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:33:39.203268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 594/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:33:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:33:39.474879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004b762-68a2-45b4-83c7-fbb24078966c None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:39.511151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:39.573763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004b762-68a2-45b4-83c7-fbb24078966c None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:39.606372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:39.606983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:39.607255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:39.607547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:39.617594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:33:39.622335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:33:39.623419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:33:39.624677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 601/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:33:39.630066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:39.725781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdf8db-f9fc-4f12-95f2-5e6698a73b82 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:39.740180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:39.761445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:39.762036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:39.762309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:39.762571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:39.763220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:39.763536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:39.764895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:39.772808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 595/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:39.778667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:39.876903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:39.884145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08683470-0032-49fe-b76d-9a7b7fde9623 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:39.915987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:39.916598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:39.916875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:39.917143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] PATH_INFO: `/v3/users/b0b0ab4e8f1e4c44b239f014be471305` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:39.918245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b0ab4e8f1e4c44b239f014be471305)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:39.919685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:39.921245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 602/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 00:33:39 2022] GET /identity/v3/users/b0b0ab4e8f1e4c44b239f014be471305 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:39.927706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:40.026620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bb957-4a25-4b7e-99f5-81cb4f6ba8fe None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:40.060132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:40.060718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:40.060993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:40.061260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:40.105193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:40.134495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:40.136743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:40.247635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 596/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:33:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:33:40.253573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:40.348950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84571e-c011-40a9-afa5-078959444b46 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:40.380716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fd5aa-62aa-496f-b1e1-52f2e09b481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:40.381332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fd5aa-62aa-496f-b1e1-52f2e09b481f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:40.381606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fd5aa-62aa-496f-b1e1-52f2e09b481f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:40.381875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fd5aa-62aa-496f-b1e1-52f2e09b481f None admin] PATH_INFO: `/v3/projects/c3bac4a7fc284b9db681d1b007904f3a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:40.383215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fd5aa-62aa-496f-b1e1-52f2e09b481f None admin] RBAC: Authorizing `identity:get_project(project_id=c3bac4a7fc284b9db681d1b007904f3a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:40.384673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fd5aa-62aa-496f-b1e1-52f2e09b481f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:40.386272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 603/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 00:33:40 2022] GET /identity/v3/projects/c3bac4a7fc284b9db681d1b007904f3a => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:33:40.392284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:33:40.471462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:40.496244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732589c-93f3-4919-826b-5bff6ba9fdf5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:33:40.530660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:33:40.531260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:33:40.531565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:33:40.531837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:33:40.532559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=c3bac4a7fc284b9db681d1b007904f3a, user.id=b0b0ab4e8f1e4c44b239f014be471305) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:33:40.532866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:33:40.534912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081ed2-110d-4bb9-bc63-910f30e76a7f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:33:40.540677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 597/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:33:40 2022] GET /identity/v3/role_assignments?user.id=b0b0ab4e8f1e4c44b239f014be471305&scope.project.id=c3bac4a7fc284b9db681d1b007904f3a&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:33:50.376622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=84 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:33:50.538077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=30 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:35.911786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1d69-98c4-4ae3-85f4-6ed898a5432e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:35.912415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1d69-98c4-4ae3-85f4-6ed898a5432e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:35.912415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1d69-98c4-4ae3-85f4-6ed898a5432e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:35.913810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 604/1201] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:34:35.923072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:35.923072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:35.923072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:35.946793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:34:35.946793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:34:36.037750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 598/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:34:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:34:36.332167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e1d69-98c4-4ae3-85f4-6ed898a5432e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:36.465301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e1d69-98c4-4ae3-85f4-6ed898a5432e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:36.509365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:36.510054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:36.510358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:36.510646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:36.524547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:36.526375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:36.562386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:36.564085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:36.568626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:36.568626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:36.568626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:36.568626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 605/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:36.570932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:36.707547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2f1c7-9020-4167-8802-c5f8992dbe7e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:36.746689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:36.754778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:36.755386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:36.755730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:36.756027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:36.756938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:36.757294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:36.758626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff10249-ff3c-4369-aff8-7201085c8c68 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:36.776967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 599/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:36.777958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:36.849820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:36.909715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23502964-1e5d-41a8-8bb3-412477a93e63 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:36.953729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840880c-eddf-4315-a8ec-c37714ff39c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:36.954432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840880c-eddf-4315-a8ec-c37714ff39c2 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:36.954661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840880c-eddf-4315-a8ec-c37714ff39c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:36.954820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840880c-eddf-4315-a8ec-c37714ff39c2 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:36.955546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840880c-eddf-4315-a8ec-c37714ff39c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:36.957211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840880c-eddf-4315-a8ec-c37714ff39c2 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:36.983529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 606/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 00:34:36 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:34:39.555144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffdffc-1ec4-4f57-80c3-829be5363409 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:39.555783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffdffc-1ec4-4f57-80c3-829be5363409 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:39.555783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffdffc-1ec4-4f57-80c3-829be5363409 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:39.557295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 600/1206] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:34:39.564881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:39.565349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:39.565708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:39.586753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:34:39.588238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:34:39.661210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 607/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:34:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:34:39.968076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffdffc-1ec4-4f57-80c3-829be5363409 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:40.092022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffdffc-1ec4-4f57-80c3-829be5363409 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:40.133057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:40.134332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:40.134332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:40.134509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:40.147514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:34:40.152842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:40.154783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:40.155657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 601/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:34:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:40.160382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:40.208283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:40.275523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c283f3a-138b-4270-b686-c3a86317641c None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:40.318729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:40.319444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:40.319753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:40.320032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:40.320764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:40.321065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:40.322517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:40.331818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 608/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:34:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:34:40.337880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:40.396906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:40.460952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67aee-1084-40b2-8e49-f524249e6ccf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:40.507439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:40.508228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:40.508488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:40.508756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:40.522015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:40.523783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:40.542263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:40.543305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:40.544266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:40.544266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:40.544266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:40.549279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 602/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:34:40.550611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:40.674705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:40.677983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a040e-5656-446f-8581-816754fa5de5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:40.716756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:40.717607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:40.717607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:40.717971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:40.718491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:40.719306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:40.720713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:40.737974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 609/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:40.739841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:40.855361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee43e8-95bb-494c-a404-ed62f9d3be89 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:40.857205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:40.894297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:40.894955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:40.895239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:40.895573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:40.903822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:40.905339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:34:40.918085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:40.919348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:40.920568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 603/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:40.926437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:41.050818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb601f7-53b8-4ef7-b652-675f8993f2f8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:41.091997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:41.092887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:41.093162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:41.093488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:41.094974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:41.095358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:41.097116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:41.116598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 610/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 00:34:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:41.122434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:41.238262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488762e0-bbc3-4861-87e0-8e751fd9ebfa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:41.281931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:41.282887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:41.283350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:41.283821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:41.321226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:41.369347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:41.370803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:41.482786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 604/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:34:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:34:41.492073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:41.513039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:41.645684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b0a11-6d4c-447e-bcc0-d3a8494eef6c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:41.693488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:41.696065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:41.696598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:41.697061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:41.714857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:41.716593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:41.738589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:41.739670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:41.740644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:41.740644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:41.740644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:41.740644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 611/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:41.745985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:41.814410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:41.855858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f68c-8302-436b-9e30-22f2f0c19e16 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:41.895972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:41.897296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:41.897655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:41.898002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:41.898736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:41.899109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:41.902932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:41.919362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 605/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:34:41.927860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:42.009786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:42.042063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97301d6b-2b1c-4094-9f45-d7f4db287960 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:42.083425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:42.084251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:42.084545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:42.084848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:42.092329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:42.093773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:34:42.105680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:42.106822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:42.107864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 612/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:42.113277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:42.226733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0827a-4c5b-4bda-90b7-1203d4eba482 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:42.252342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:42.267467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:42.268107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:42.268384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:42.268654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:42.269345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:42.269649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:42.271084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cceca-0098-46af-8a4a-57952ef64640 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:42.284688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 606/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:42.290554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:42.401338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127dd94-cbf2-4271-902d-f99c9befc013 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:42.432679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:42.439103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:42.439798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:42.440175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:42.440505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:42.441376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=ce70c358cf0f46139f419cf894a97fcf) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:42.441681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:42.443341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29749a-896a-4a94-8f44-19a623e20db3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:42.449918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 613/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:34:42 2022] GET /identity/v3/role_assignments?user.id=ce70c358cf0f46139f419cf894a97fcf&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:44.596262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef32e6-7229-46f5-8a39-209d1610f438 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:44.596816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef32e6-7229-46f5-8a39-209d1610f438 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:44.596816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef32e6-7229-46f5-8a39-209d1610f438 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:44.598172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 607/1220] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:34:44.603377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:44.603812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:44.603960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:44.623005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:34:44.624259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:34:44.695185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 614/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:34:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:34:45.025227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef32e6-7229-46f5-8a39-209d1610f438 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:45.139396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef32e6-7229-46f5-8a39-209d1610f438 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:45.178732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:45.179380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:45.179680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:45.179944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:45.191176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:34:45.195979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:45.197005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:45.197939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 608/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:34:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:45.204073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:45.318777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf882a-cf2b-481a-83f3-625bd1cc70a6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:45.346736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:45.358375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:45.359588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:45.359922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:45.360193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:45.360847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:45.361151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:45.362741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:45.371930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 615/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:34:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:45.378318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:45.426324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:45.486409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae5613-5fa4-4274-be06-ae1e9dc86451 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:45.520873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:45.521514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:45.521797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:45.522049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:45.532628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:45.534071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:45.549776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:45.550937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:45.552342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:45.552342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:45.552342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:45.552342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 609/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:45.556962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:45.670283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67649d93-0a8e-4bd9-9290-f79bd6d47b02 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:45.706624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:45.707302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:45.707712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:45.707969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:45.708697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:45.709027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:45.710507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:45.723102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 616/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:34:45.728962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:45.830724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:45.838406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a752-eb59-4c9b-880d-94d495df347e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:45.876362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:45.876980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:45.877255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:45.877526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:45.885350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:45.886774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:34:45.899549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:45.900604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:45.902141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 610/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:45.907279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:45.958989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:46.016175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1851db-3a46-400f-9ca3-7cdda68355f8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:46.053483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:46.054589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:46.054812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:46.055101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:46.055838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:46.056140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:46.057556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:46.075232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 617/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 00:34:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:46.082514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:46.190010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb49e5d-cf8b-42dc-93cb-04fdd17f2da6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:46.223935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:46.224477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:46.224750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:46.225015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:46.235038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:46.236511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:46.250540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:46.251853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:46.253200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:46.253200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:46.253200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:46.253200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 611/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:34:46.257667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:46.372688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abab31-4482-4fd0-97ed-bd09eccffbf5 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:46.375921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:46.408338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:46.408919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:46.409201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:46.409471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:46.410112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:46.410416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:46.411805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:46.424590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 618/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:46.430121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:46.440040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:46.535095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4a7bc-f1f7-4b44-b489-f911f2c4fbf1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:46.575625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:46.576253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:46.576535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:46.576806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:46.583847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:46.585401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:34:46.596853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:46.597819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:46.598824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 612/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:34:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:46.604234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:46.716423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0cb40-4a28-4cd3-a731-95e3adea586b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:46.752344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:46.753020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:46.753296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:46.753634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:46.754318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:46.754636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:46.756153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c1bb-8c43-4b7a-b8f2-22dc6cf64f74 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:46.769347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 619/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:34:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:46.775317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:46.849070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:46.888069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b5a38-794b-464b-b3b0-12c038267469 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:46.923103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68122bbe-2dec-4862-aa75-367decfaedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:46.924103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68122bbe-2dec-4862-aa75-367decfaedf9 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:46.924432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68122bbe-2dec-4862-aa75-367decfaedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:46.924735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68122bbe-2dec-4862-aa75-367decfaedf9 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/ce70c358cf0f46139f419cf894a97fcf/roles/150cf3664228468f8e9bd9251ac51350` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:46.941502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68122bbe-2dec-4862-aa75-367decfaedf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=ce70c358cf0f46139f419cf894a97fcf, role_id=150cf3664228468f8e9bd9251ac51350)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:46.943172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68122bbe-2dec-4862-aa75-367decfaedf9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:46.957811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 613/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:34:46 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/ce70c358cf0f46139f419cf894a97fcf/roles/150cf3664228468f8e9bd9251ac51350 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:34:49.198189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a85efe-0b34-4926-921f-e339a34ac242 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:49.198757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a85efe-0b34-4926-921f-e339a34ac242 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:49.198757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a85efe-0b34-4926-921f-e339a34ac242 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:49.200095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 620/1233] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:34:49.207172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:49.207646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:49.207743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:49.227272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:34:49.228740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:34:49.311642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 614/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:34:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:34:49.604561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a85efe-0b34-4926-921f-e339a34ac242 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:49.632074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:49.724553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a85efe-0b34-4926-921f-e339a34ac242 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:49.764658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:49.765254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:49.765515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:49.765777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:49.776417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:34:49.782227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:49.787522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:49.788727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 621/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 00:34:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:49.790993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:49.840421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:49.900732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4baca8-4ce3-4b33-8eb4-29fa193a14ab None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:49.936893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:49.937498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:49.937787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:49.938035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:49.938690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:49.938986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:49.940380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:49.948476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 615/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 00:34:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:49.954454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:50.047546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:50.065772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f2ca1-7bd0-47a7-8c4c-2c8f9dbc7b8d None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:50.103499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:50.104247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:50.104587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:50.104859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:50.116393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:50.118187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:50.136197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:50.137497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:50.138516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:50.138516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:50.138516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:50.138516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 622/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:50.143900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:50.247115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e379e3-697a-41b3-8cd2-fff8b63d6b09 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:50.257713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:50.289126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:50.289751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:50.290096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:50.290359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:50.291243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:50.291533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:50.293262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:50.308605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 616/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:50.315013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:50.425946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a871f-a242-4970-a60a-6dd226e00c96 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:50.463979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:50.464840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:50.465220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:50.465525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:50.469088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:50.474738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:50.476362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:34:50.490741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:50.491878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:50.492857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 623/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:50.498313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:50.606575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafc50c-2a41-4128-8008-1e93fe1a7e84 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:50.645354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:50.646017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:50.646301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:50.646582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:50.647315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:50.647663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:50.649549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:50.669663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 617/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 00:34:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:50.676494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:50.787658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1b723-96e7-41e3-a7b5-fddb9a67457b None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:50.822012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:50.822616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:50.822892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:50.823196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:50.890823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:50.899107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:50.900942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:51.005682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 624/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:34:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:34:51.012533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:51.042696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:51.128975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7727-b099-4603-8793-759f915fd33a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:51.170433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:51.171102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:51.171417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:51.171818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:51.184391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:51.186306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:51.203939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:51.205321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:51.206670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:51.206670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:51.206670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:51.206670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 618/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:51.211938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:51.329669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:51.337753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220052d-cf13-4e75-9172-face0e08d13a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:51.382753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:51.383454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:51.383783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:51.384114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:51.384867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:51.385202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:51.386764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:51.400233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 625/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:51.406658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:51.490197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:51.513359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46653d87-0774-4b52-9de0-80fd228c6102 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:51.549087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:51.549706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:51.549982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:51.550269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:51.557214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:51.558737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:34:51.570241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:51.571614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:51.572851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 619/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:51.577961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:51.692944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51ff97-0d6b-49a6-aeba-161cfc74936c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:51.737435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:51.738046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:51.738311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:51.738573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:51.739266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:51.739583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:51.741033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bef84d-978e-45c9-89d7-c765495c701f None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:51.753205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 626/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:51.759670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:51.874946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a6c58-de3f-423e-aaa0-b9fcf1d2ecdd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:51.915253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:51.915909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:51.916179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:51.916444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:51.917165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] RBAC: Adding query filter params (role.id=150cf3664228468f8e9bd9251ac51350, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=ce70c358cf0f46139f419cf894a97fcf) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:51.917505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:51.918972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78976644-7855-4227-9e67-8fd942c1f91d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:51.924979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 620/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:34:51 2022] GET /identity/v3/role_assignments?user.id=ce70c358cf0f46139f419cf894a97fcf&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=150cf3664228468f8e9bd9251ac51350 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:54.489515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071b040-d88e-44e6-8875-7ae1d244c38f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:54.490080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071b040-d88e-44e6-8875-7ae1d244c38f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:54.490227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071b040-d88e-44e6-8875-7ae1d244c38f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:54.491970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 627/1247] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:34:54.498319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:54.498740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:54.498890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:54.505290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:54.518976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:34:54.520229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:34:54.594830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 621/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:34:54.861439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071b040-d88e-44e6-8875-7ae1d244c38f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:54.879931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:54.980608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071b040-d88e-44e6-8875-7ae1d244c38f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:55.022470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:55.023202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:55.023644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:55.024105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:55.037496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:34:55.042764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:55.043767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:55.044782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 628/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:34:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 00:34:55.050571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:55.156032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c45bb-3343-4b42-bfa9-7ad28ab4e133 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:55.190423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:55.196395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:55.197109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:55.197471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:55.197802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:55.198629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:55.198947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:55.200390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:55.209972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 622/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:55.216228 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:55.307069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:55.337472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7be77-58fa-42cd-b50c-d56542328655 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:55.382285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:55.382923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:55.383202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:55.383464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:55.394479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:55.395982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:55.411815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:55.417829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:55.418959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:55.418959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:55.418959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:55.418959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 629/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:55.419247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:55.523422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78007-4637-4979-b2a5-b72031fb561d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:55.559611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:55.560219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:55.560492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:55.560757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:55.561406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:55.561741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:55.563136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:55.581234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 623/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:34:55.582464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:55.687287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfa3b3-812c-4fa6-bae6-116b46399699 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:55.714137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:55.722487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:55.723114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:55.723389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:55.723688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:55.731429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:55.732888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:34:55.744070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:55.745169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:55.746336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 630/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:55.751659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:55.808615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:55.853271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17f427-b807-4756-b575-1bff8f670879 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:55.889241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:55.890430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:55.890705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:55.890973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:55.891765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:55.892070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:55.893507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:55.910830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 624/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 00:34:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:55.916588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:56.017966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c04ac-e438-4b17-9e92-6b4b98e42335 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:56.049308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:56.049893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:56.050169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:56.050436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:56.102466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:56.115011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:56.116469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:56.213707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 631/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:34:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:34:56.219506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:56.320964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d545e-3dd0-4e7f-97f5-cfbd1f66a42e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:56.337196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:56.359460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:56.360112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:56.360387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:56.360654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:56.370308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:56.371780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:56.386690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:56.387818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:56.388922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:56.388922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:56.388922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:56.388922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 625/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:56.393575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:56.476008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:56.488152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90726-55e2-4ef5-af3b-c2f154e7e44e None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:56.520153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:56.520740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:56.521017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:56.521291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:56.521920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:56.522217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:56.523579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:56.534867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 632/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:56.540379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:56.637327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f12f7-7051-44de-b79a-b5bbff9f3e27 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:56.669503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:56.670112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:56.670386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:56.670650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:56.677645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:56.679084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:34:56.689948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:56.690970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:34:56.696536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 626/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:34:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:56.697023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:56.791274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112ac33-2094-453d-8def-f3338a89a911 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:56.822799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:56.823379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:56.823683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:56.823955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:56.824625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:56.824922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:56.826311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531daa1-27f2-4b09-a63f-212927d07a65 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:56.838057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 633/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:34:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:56.844433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:56.869053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:56.946979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91d12-bdb8-4d81-8f6d-247bbdce829f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:56.979734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:56.980320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:56.980594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:56.980860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:56.981573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=ce70c358cf0f46139f419cf894a97fcf) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:56.981899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:56.983386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b118839-5ebf-4712-a137-ad73baeb1d47 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:56.988637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 627/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:34:56 2022] GET /identity/v3/role_assignments?user.id=ce70c358cf0f46139f419cf894a97fcf&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:34:59.093539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8721e-5af1-42ff-bbf6-5ba39222ccdc None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:59.094056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8721e-5af1-42ff-bbf6-5ba39222ccdc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:59.094160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8721e-5af1-42ff-bbf6-5ba39222ccdc None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:59.095421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 634/1261] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:34:59.100438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:59.100816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:59.100983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:59.118528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:34:59.119741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:34:59.183535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 628/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:34:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:34:59.448848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8721e-5af1-42ff-bbf6-5ba39222ccdc None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:59.459612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:59.548201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8721e-5af1-42ff-bbf6-5ba39222ccdc None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:59.580063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:59.580678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:59.580955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:59.581224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:59.590892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:34:59.596431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:59.597531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:59.598439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 635/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:34:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:59.603494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:59.621761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:59.699610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7241-ebc0-4810-b4b4-7f0ed0d4fda2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:59.731656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:59.732249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:59.732525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:59.732793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:59.734030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:34:59.734326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:59.735700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:59.743346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 629/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:34:59.749050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:59.813451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:34:59.845453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f303c75-46a2-4c93-9a65-80a18147362c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:34:59.877052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:34:59.877656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:34:59.877933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:34:59.878204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:34:59.887985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:34:59.890030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:34:59.903967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:34:59.905589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:34:59.906712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:34:59.906712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:34:59.906712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:34:59.909291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 636/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:34:59.911115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:34:59.988140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:00.006844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bebff-3146-476f-b9d6-84aec460ee59 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.039605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.040187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.040462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.040732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.041369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:00.041668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.043013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.054352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 630/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:00.060048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:00.156546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0febac-4e48-430b-96d4-20b6a06df530 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.175110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:00.189259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.189875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.190151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.190421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.197508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.198917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:35:00.210270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:00.211354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:35:00.212702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 637/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:35:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:00.217143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:00.318167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7d76-8659-40e6-b5db-977189a6f458 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.353380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.353973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.354251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.354520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.355239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:00.355574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.356994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.365267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:00.373415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 631/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 00:35:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:00.379701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:00.474284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371876-f923-4e36-97b6-73907adf2df8 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.506377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.506980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.507258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.507554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.518153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.519602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:35:00.533102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:00.534150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:35:00.535587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:35:00.535587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:35:00.535587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:35:00.535587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 638/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:00.539597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:00.634832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfda94-d490-427a-9129-ec7ec06a07a4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.667810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.668396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.668670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.668934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.669567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:00.669863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.671735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.684265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 632/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:00.690588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:00.721122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:00.785864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe037d-ab81-4d3e-84d6-8d010cd0bac9 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.817941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.818548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.818817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.819080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.825490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.826905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:35:00.837697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:00.838735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:35:00.839843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 639/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:00.844532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:00.900723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:00.940672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ed6f1-e074-441a-82f9-126cabda4384 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:00.971975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:00.972554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:00.972826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:00.973091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:00.973760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:00.974056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:00.975510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f170-d635-4625-a570-0fa44a80c7e1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:00.987312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 633/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:35:00.993156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:01.079150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:01.087678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899ef9-01d6-45e9-acfc-699f6e882fc0 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:01.119752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d9b1f-adad-4624-bcb9-35ce2fe46193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:01.120379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d9b1f-adad-4624-bcb9-35ce2fe46193 None admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:01.120650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d9b1f-adad-4624-bcb9-35ce2fe46193 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:01.120914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d9b1f-adad-4624-bcb9-35ce2fe46193 None admin] PATH_INFO: `/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/ce70c358cf0f46139f419cf894a97fcf/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:01.122748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d9b1f-adad-4624-bcb9-35ce2fe46193 None admin] RBAC: Authorizing `identity:create_grant(project_id=217beeb34c094465ace3f6e1b3c979a4, user_id=ce70c358cf0f46139f419cf894a97fcf, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:01.124200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d9b1f-adad-4624-bcb9-35ce2fe46193 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:01.136222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 640/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 00:35:00 2022] PUT /identity/v3/projects/217beeb34c094465ace3f6e1b3c979a4/users/ce70c358cf0f46139f419cf894a97fcf/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:35:03.224299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148d893-3261-4ed6-8a18-e7d4d9c06098 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:03.224824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148d893-3261-4ed6-8a18-e7d4d9c06098 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:03.224824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148d893-3261-4ed6-8a18-e7d4d9c06098 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:03.226167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 634/1274] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:35:03.231304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:03.231726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:03.231888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:03.250219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:35:03.251382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:35:03.320186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 641/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:35:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:35:03.587247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148d893-3261-4ed6-8a18-e7d4d9c06098 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:03.686213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148d893-3261-4ed6-8a18-e7d4d9c06098 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:03.703023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:03.719664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:03.720275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:03.720549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:03.720814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:03.730382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 00:35:03.735118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:03.736222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:03.737104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 635/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 00:35:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:03.742177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:03.819585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:03.838485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753d6c1-28b9-4730-b87d-64653e1a5db4 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:03.870594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:03.871184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:03.871459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:03.871763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:03.872752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:03.873049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:03.874426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:03.882318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 642/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 00:35:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:03.888252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:03.984699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b99a03-7801-4cd9-9b7d-d4ba2e32c891 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:04.017754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:04.018348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:04.018620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:04.018887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:04.028462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:04.029882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:35:04.043808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:04.044929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:35:04.046027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:35:04.046027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:35:04.046027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:04.046027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 636/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:04.050389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:04.146785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85166bd8-be5b-430c-886c-540144ae98a4 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:04.178158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:04.178743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:04.179015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:04.179279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:04.179933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:04.180227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:04.181563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:04.193146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 643/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:04.198774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:04.228307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:04.295203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaba7f-104c-4286-a81d-e20446cb21ce None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:04.329305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:04.329920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:04.330197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:04.330472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:04.338340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:04.339816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 00:35:04.350589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:04.351701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:04.352998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 637/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:35:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:04.357571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:04.361903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:04.453520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85bdd07-5602-4345-8224-edf532caf03a None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:04.485443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:04.486030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:04.486307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:04.486576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:04.487290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:04.487624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:04.489052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:04.505813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 644/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 00:35:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:04.511822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:04.596813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:04.606351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c805f-5aa8-4f8a-8176-d62e1d2ec36a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:04.638726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:04.639503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:04.639837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:04.640104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:04.706163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:04.707578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:04.790434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:04.801000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 638/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 00:35:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 00:35:04.806900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:04.865301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:04.901984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde78fc-eb10-4560-8265-0d67465625f3 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:04.934318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:04.934924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:04.935195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:04.935459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:04.944942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:04.946352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 00:35:04.960307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:04.961226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 00:35:04.962220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 00:35:04.962220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 00:35:04.962220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:35:04.962220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 645/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:04.967106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:05.061428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d80231-fca4-4c3c-9920-5a89b46e6a4c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:05.094366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:05.094992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:05.095269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:05.095568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:05.096236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:05.096534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:05.097882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:05.108739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 639/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 00:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:05.114157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:05.210856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34dde1-2f19-47fe-a673-1244ea50c559 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:05.224852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:05.244374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:05.244986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:05.245259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:05.245521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:05.251874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:05.253290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 00:35:05.264845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:05.265895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: ERROR keystone.server.flask.application Sep 15 00:35:05.266993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 646/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 00:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:05.271770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:05.325290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:05.375647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb5e3-987b-4325-91a9-17b8453b35ab None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:05.409196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:05.409799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:05.410074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:05.410344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:05.411034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:05.411338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:05.412809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41082d5-d001-4a44-a14d-6a6217ec72c8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:05.424461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 640/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 00:35:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:05.430505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:05.526580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a096fe-dd93-40ef-907c-171bd2777d7c None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:05.557900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:05.558488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:05.558762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:05.559030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:05.559770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] RBAC: Adding query filter params (role.id=971eb6603a5a4b8a80d419690dfbeb95, scope.project.id=217beeb34c094465ace3f6e1b3c979a4, user.id=ce70c358cf0f46139f419cf894a97fcf) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:05.560074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:05.562053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24338a2-5d48-4f97-8864-78aa39957d58 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:05.567786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 647/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 00:35:05 2022] GET /identity/v3/role_assignments?user.id=ce70c358cf0f46139f419cf894a97fcf&scope.project.id=217beeb34c094465ace3f6e1b3c979a4&role.id=971eb6603a5a4b8a80d419690dfbeb95 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:07.723467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d4a5-b650-4fb0-a7a0-62a226153967 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:07.723980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d4a5-b650-4fb0-a7a0-62a226153967 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:07.724092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d4a5-b650-4fb0-a7a0-62a226153967 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:07.725363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 641/1288] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:35:07.730398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:07.730761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:07.730954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:07.749298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:35:07.750474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:35:07.778449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:07.814928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 648/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:35:08.078356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d4a5-b650-4fb0-a7a0-62a226153967 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:08.179613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d4a5-b650-4fb0-a7a0-62a226153967 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:08.190714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:08.214462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:08.215079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:08.215357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:08.215657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:08.216233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:08.218288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:08.226966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:08.228211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:08.229254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 642/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:35:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:08.234113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:08.333239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8bd79-cb5a-431c-91cc-6341ad8f89d9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:08.375074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:08.375705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:08.375982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:08.376251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:08.376880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:08.377179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:08.378538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609802-b990-4c94-be3a-fc0d74bf989e None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:08.383499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 649/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 00:35:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:08.389277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:08.486055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7683c-a0b0-4f66-8427-7c2b34cc120f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:08.517987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:08.518566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:08.518842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:08.519114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:08.519776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:08.520078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:08.521435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126670e-1142-456f-b5e4-dc3a82585a22 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:08.525660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 643/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 00:35:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:10.634765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dc2e6-052b-43bf-8bc9-c6219a24602f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:10.635313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dc2e6-052b-43bf-8bc9-c6219a24602f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:10.635313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dc2e6-052b-43bf-8bc9-c6219a24602f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:10.636662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 650/1293] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:35:10.642419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a66205-21c2-4dc5-a54c-39624bd81851 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:10.642830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a66205-21c2-4dc5-a54c-39624bd81851 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:10.642988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a66205-21c2-4dc5-a54c-39624bd81851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:10.661235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c7a66205-21c2-4dc5-a54c-39624bd81851 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:35:10.662405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c7a66205-21c2-4dc5-a54c-39624bd81851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:35:10.693074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:10.728797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 644/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:35:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:35:10.992568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dc2e6-052b-43bf-8bc9-c6219a24602f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:11.023683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:11.090497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dc2e6-052b-43bf-8bc9-c6219a24602f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:11.122466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce6247-9b42-4f17-9f34-8daadb36c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:11.123045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6247-9b42-4f17-9f34-8daadb36c515 None admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:11.123898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6247-9b42-4f17-9f34-8daadb36c515 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:11.124166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6247-9b42-4f17-9f34-8daadb36c515 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:11.124803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce6247-9b42-4f17-9f34-8daadb36c515 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:11.126227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce6247-9b42-4f17-9f34-8daadb36c515 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:11.140500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 651/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 00:35:10 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:35:13.290559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede4230-6b50-45f9-a889-afe7c047d45a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:13.291083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede4230-6b50-45f9-a889-afe7c047d45a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:13.291083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede4230-6b50-45f9-a889-afe7c047d45a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:13.292639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 645/1296] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:35:13.298642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:13.299123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:13.299237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:13.319913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:35:13.321284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:35:13.399842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 652/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:35:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:35:13.664267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede4230-6b50-45f9-a889-afe7c047d45a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:13.762752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede4230-6b50-45f9-a889-afe7c047d45a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:13.795592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:13.796204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:13.796484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:13.796755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:13.797358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:13.798715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:13.807235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:13.808323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:13.809547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 646/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:35:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:13.814361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:13.832115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:13.910629 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d95f2-62d5-4742-b77b-a3f7a50a109d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:13.943354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:13.943917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:13.944198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:13.944470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:13.945094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:13.945396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:13.946753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:13.951768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 653/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 00:35:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:13.957613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:13.988704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:14.055857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13f39e-8d20-427d-b0e5-b66177bd9267 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:14.087937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:14.088519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:14.088794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:14.089062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:14.089755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] RBAC: Adding query filter params (interface=public, service_id=b4e16501d9f745349456a2322d2a1ce1, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:14.090058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:14.091410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabbaae-8aeb-4954-a877-cf440a0f5dc2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:14.096804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 647/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 00:35:13 2022] GET /identity/v3/endpoints?service_id=b4e16501d9f745349456a2322d2a1ce1&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:14.102352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:14.179689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:14.200587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f73c0b-5f48-468f-b711-023b43871783 None admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:14.233323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a970-fdcc-4399-b57b-1acfee369d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:14.233903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a970-fdcc-4399-b57b-1acfee369d0b None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:14.234176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a970-fdcc-4399-b57b-1acfee369d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:14.234442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a970-fdcc-4399-b57b-1acfee369d0b None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:14.235034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a970-fdcc-4399-b57b-1acfee369d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:14.236398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a970-fdcc-4399-b57b-1acfee369d0b None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:14.242852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 654/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 00:35:14 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:35:16.377535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0161f1f-9e9f-40e0-adc0-14bd59a4aa5b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:16.378099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0161f1f-9e9f-40e0-adc0-14bd59a4aa5b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:16.378099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0161f1f-9e9f-40e0-adc0-14bd59a4aa5b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:16.379385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 648/1302] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:35:16.384715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:16.385132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:16.385242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:16.402831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:35:16.404036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:35:16.467844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 655/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:35:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 00:35:16.733570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0161f1f-9e9f-40e0-adc0-14bd59a4aa5b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:16.826650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:16.833567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0161f1f-9e9f-40e0-adc0-14bd59a4aa5b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:16.866594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:16.867208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:16.867511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:16.867777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:16.868353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:16.869711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 00:35:16.878333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 00:35:16.879637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Sep 15 00:35:16.881022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 649/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 00:35:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 00:35:16.885629 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:16.957364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:16.979417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781e29b-04b6-46fa-8770-5c7578ab2bc3 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:17.012178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:17.012762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:17.013036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:17.013304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] PATH_INFO: `/v3/services` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:17.013922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:35:17.014265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:17.016047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e037f5-54cb-48de-a8d5-dd9d38ae1831 None admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:17.021016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 656/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 00:35:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:35:17.026603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:35:17.122196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ec6a7-a95d-463f-9bcb-7ac9fdfede5a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:35:17.154626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc7004-a845-42ac-929d-ad11128f491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:35:17.155753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc7004-a845-42ac-929d-ad11128f491d None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:35:17.156028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc7004-a845-42ac-929d-ad11128f491d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:35:17.156294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc7004-a845-42ac-929d-ad11128f491d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:35:17.156952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc7004-a845-42ac-929d-ad11128f491d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:35:17.158295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc7004-a845-42ac-929d-ad11128f491d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:35:17.176645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 650/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:35:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:35:27.018200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=30 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:35:27.173175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:21.047560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160163b-2c8b-4e06-9456-80c2327ed380 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:21.048039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160163b-2c8b-4e06-9456-80c2327ed380 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:21.048127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160163b-2c8b-4e06-9456-80c2327ed380 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:21.056200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7234ea-790e-4034-bb0e-1241d3883690 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:21.056592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7234ea-790e-4034-bb0e-1241d3883690 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:21.056677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7234ea-790e-4034-bb0e-1241d3883690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:21.059352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 657/1307] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:21.143348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3e7234ea-790e-4034-bb0e-1241d3883690 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:21.144780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3e7234ea-790e-4034-bb0e-1241d3883690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:21.278498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 651/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:21.571818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868bff8-401e-4b19-93d5-10aae785f218 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:21.572125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868bff8-401e-4b19-93d5-10aae785f218 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:21.572671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868bff8-401e-4b19-93d5-10aae785f218 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:21.574542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 658/1309] 10.222.0.15 () {54 vars in 965 bytes} [Thu Sep 15 00:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:21.580696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eee1a-4668-47bc-b659-4cf21c48342a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:21.581054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eee1a-4668-47bc-b659-4cf21c48342a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:21.581420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eee1a-4668-47bc-b659-4cf21c48342a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:21.667248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ad0eee1a-4668-47bc-b659-4cf21c48342a None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:21.670932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ad0eee1a-4668-47bc-b659-4cf21c48342a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:21.691241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:21.751445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 652/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Thu Sep 15 00:36:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:21.760600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868bff8-401e-4b19-93d5-10aae785f218 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:21.763016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7868bff8-401e-4b19-93d5-10aae785f218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:21.882205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868bff8-401e-4b19-93d5-10aae785f218 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:21.962457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:21.963069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:21.963344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:21.963644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:21.965270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:22.023848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:22.090343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:22.091846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053240b-335f-4944-b110-b90609cd81bb service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:22.113253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 659/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:24.576970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8cba8-cd32-4f52-9367-b296f8eaa6b1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:24.577482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8cba8-cd32-4f52-9367-b296f8eaa6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:24.577569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8cba8-cd32-4f52-9367-b296f8eaa6b1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:24.578858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 653/1312] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:24.585534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729084-303b-431d-aac3-be1f4db9d4ec None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:24.585973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729084-303b-431d-aac3-be1f4db9d4ec None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:24.586068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729084-303b-431d-aac3-be1f4db9d4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:24.670345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1e729084-303b-431d-aac3-be1f4db9d4ec None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:24.671598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1e729084-303b-431d-aac3-be1f4db9d4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:24.766059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 660/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:25.062998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496644e-1d33-454c-80b3-2bb32a1f4eb8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:25.063282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496644e-1d33-454c-80b3-2bb32a1f4eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:25.063596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496644e-1d33-454c-80b3-2bb32a1f4eb8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:25.064943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 654/1314] 10.222.0.15 () {54 vars in 965 bytes} [Thu Sep 15 00:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:25.071119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235552-2e1e-4e86-92c1-6e2d1834d436 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:25.071474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235552-2e1e-4e86-92c1-6e2d1834d436 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:25.071842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235552-2e1e-4e86-92c1-6e2d1834d436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:25.084126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:25.155468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ff235552-2e1e-4e86-92c1-6e2d1834d436 None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:25.156850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ff235552-2e1e-4e86-92c1-6e2d1834d436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:25.237083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 661/1315] 10.222.0.15 () {58 vars in 1062 bytes} [Thu Sep 15 00:36:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:25.246897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496644e-1d33-454c-80b3-2bb32a1f4eb8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:25.248491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c496644e-1d33-454c-80b3-2bb32a1f4eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:25.356388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496644e-1d33-454c-80b3-2bb32a1f4eb8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:25.428162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:25.430719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:25.431368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:25.431850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:25.432290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:25.434729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:25.541221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:25.543108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c23ccb-234d-4259-a781-3afcc9238915 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:25.562153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 655/1316] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:36:28.241921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c98b-c5ce-4e62-850e-344ee7b03738 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:28.242384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c98b-c5ce-4e62-850e-344ee7b03738 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:28.242477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c98b-c5ce-4e62-850e-344ee7b03738 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:28.243865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 662/1317] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:28.249041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7754dc-7175-4ddf-a7f4-b4a806b6f9d9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:28.249470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7754dc-7175-4ddf-a7f4-b4a806b6f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:28.249590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7754dc-7175-4ddf-a7f4-b4a806b6f9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:28.315161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7b7754dc-7175-4ddf-a7f4-b4a806b6f9d9 None None] MFA Rules not processed for user `ce85ea6bdfa54b3298ecf834a6d7fd47`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:28.316405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7b7754dc-7175-4ddf-a7f4-b4a806b6f9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:28.367058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:28.411095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 656/1318] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:28.693636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386c98b-c5ce-4e62-850e-344ee7b03738 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:28.695179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f386c98b-c5ce-4e62-850e-344ee7b03738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:28.801583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386c98b-c5ce-4e62-850e-344ee7b03738 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:28.804433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:28.875371 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:28.875992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:28.876267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:28.876535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:28.878133 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:28.982603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:28.983979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71c520-01fb-4f9f-bc1c-d7f1ff7975af service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:29.001737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 663/1319] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:31.663443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b222-97c0-471b-9022-ea0ebc91d74e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:31.663994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b222-97c0-471b-9022-ea0ebc91d74e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:31.663994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b222-97c0-471b-9022-ea0ebc91d74e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:31.665417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 657/1320] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:31.671265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76410b46-0053-453b-bf43-0b3eaeb695ed None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:31.671741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76410b46-0053-453b-bf43-0b3eaeb695ed None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:31.671839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76410b46-0053-453b-bf43-0b3eaeb695ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:31.710767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:31.736622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-76410b46-0053-453b-bf43-0b3eaeb695ed None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:31.737810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-76410b46-0053-453b-bf43-0b3eaeb695ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:31.829909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 664/1321] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:32.115665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487b222-97c0-471b-9022-ea0ebc91d74e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:32.117216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4487b222-97c0-471b-9022-ea0ebc91d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:32.218995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487b222-97c0-471b-9022-ea0ebc91d74e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:32.290574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:32.291164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:32.291436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:32.292212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:32.293838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:32.300707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:32.402131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:32.403496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26db0d2-d0f0-484b-8473-df01382a30c5 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:32.424582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 658/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:35.193658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a3879-d45c-4b00-8aba-c726595048dd None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:35.194221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a3879-d45c-4b00-8aba-c726595048dd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:35.194221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a3879-d45c-4b00-8aba-c726595048dd None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:35.195643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 665/1323] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:35.201120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501464f-1f1f-4a21-bab1-e17533e35eac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:35.201547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501464f-1f1f-4a21-bab1-e17533e35eac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:35.201684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501464f-1f1f-4a21-bab1-e17533e35eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:35.271417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e501464f-1f1f-4a21-bab1-e17533e35eac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:35.272695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e501464f-1f1f-4a21-bab1-e17533e35eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:35.337480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:35.380043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 659/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:35.662941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a3879-d45c-4b00-8aba-c726595048dd None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:35.664539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c02a3879-d45c-4b00-8aba-c726595048dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:35.766858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a3879-d45c-4b00-8aba-c726595048dd None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:35.780796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:35.845329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:35.845931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:35.846207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:35.846475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:35.848091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:35.958840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:35.960221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229df45-7b69-46db-afdb-dfca8e898341 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:35.979291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 666/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:38.189954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899824-017e-414b-b34e-d25a86f21ae8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:38.190476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899824-017e-414b-b34e-d25a86f21ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:38.190476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899824-017e-414b-b34e-d25a86f21ae8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:38.191885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 660/1326] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:38.197029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a453c72-2de6-4ef9-8399-7d9473819d88 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:38.197841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a453c72-2de6-4ef9-8399-7d9473819d88 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:38.198112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a453c72-2de6-4ef9-8399-7d9473819d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:38.245821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:38.264921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4a453c72-2de6-4ef9-8399-7d9473819d88 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:38.266110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4a453c72-2de6-4ef9-8399-7d9473819d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:38.369388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 667/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:38.649711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9899824-017e-414b-b34e-d25a86f21ae8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:38.651254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c9899824-017e-414b-b34e-d25a86f21ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:38.763818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9899824-017e-414b-b34e-d25a86f21ae8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:38.839579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:38.840178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:38.840453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:38.840721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:38.842356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:38.859764 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:38.953437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:38.954790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff78a8-6ba3-45ce-adee-9fbaad158225 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:38.973380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 661/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:41.170765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc917000-9342-4221-8ce4-717637d7c668 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:41.171250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc917000-9342-4221-8ce4-717637d7c668 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:41.171250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc917000-9342-4221-8ce4-717637d7c668 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:41.172674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 668/1329] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:41.180268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b2271-c118-41f9-b033-b09925d753d4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:41.180843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b2271-c118-41f9-b033-b09925d753d4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:41.180843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b2271-c118-41f9-b033-b09925d753d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:41.246493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-547b2271-c118-41f9-b033-b09925d753d4 None None] MFA Rules not processed for user `ce85ea6bdfa54b3298ecf834a6d7fd47`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:41.247656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-547b2271-c118-41f9-b033-b09925d753d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:41.312037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:41.346327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 662/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:41.630707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc917000-9342-4221-8ce4-717637d7c668 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:41.632345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bc917000-9342-4221-8ce4-717637d7c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:41.743497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc917000-9342-4221-8ce4-717637d7c668 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:41.764087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:41.819805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:41.820410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:41.820687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:41.820956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:41.823044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:41.929688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:41.931114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d714f-b768-4bc0-af7b-1dedb362335c service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:41.949036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 669/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:44.527049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1d21-c522-4ab2-a639-31889d6610db None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:44.527689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1d21-c522-4ab2-a639-31889d6610db None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:44.527689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1d21-c522-4ab2-a639-31889d6610db None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:44.528896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 663/1332] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:36:44.534300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867aef1-c1c4-4382-bbd6-8435b249a0c1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:44.534727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867aef1-c1c4-4382-bbd6-8435b249a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:44.534846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867aef1-c1c4-4382-bbd6-8435b249a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:44.596772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:44.599500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a867aef1-c1c4-4382-bbd6-8435b249a0c1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:44.601315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a867aef1-c1c4-4382-bbd6-8435b249a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:44.696480 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 670/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:36:44.987046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d1d21-c522-4ab2-a639-31889d6610db None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:44.988609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-767d1d21-c522-4ab2-a639-31889d6610db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:45.088836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d1d21-c522-4ab2-a639-31889d6610db None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:45.162992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:45.163613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:45.163886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:45.164148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:45.165735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:45.192166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:45.274072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:45.276016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7661f97-fadd-4dee-8734-b1f2d4b3ecae service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:45.294648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 664/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:47.445509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72aa9aa-13e0-4628-9e1e-2263eb7a252b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:47.446069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72aa9aa-13e0-4628-9e1e-2263eb7a252b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:47.446069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72aa9aa-13e0-4628-9e1e-2263eb7a252b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:47.447419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 671/1335] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:47.452521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328508e-a293-4f9b-b1dc-2e8c00a44a1b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:47.452905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328508e-a293-4f9b-b1dc-2e8c00a44a1b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:47.453070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328508e-a293-4f9b-b1dc-2e8c00a44a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:47.517368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2328508e-a293-4f9b-b1dc-2e8c00a44a1b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:47.518567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2328508e-a293-4f9b-b1dc-2e8c00a44a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:47.592369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:47.614825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 665/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:47.904002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72aa9aa-13e0-4628-9e1e-2263eb7a252b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:47.905547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a72aa9aa-13e0-4628-9e1e-2263eb7a252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:48.005301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72aa9aa-13e0-4628-9e1e-2263eb7a252b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:48.033474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:48.078020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:48.078618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:48.078894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:48.079163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:48.080805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:48.187469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:48.188916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a81fab-211c-4664-878d-b25e8776b606 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:48.206758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 672/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:50.670506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5f602-f3c2-42ef-9d96-ae7a6f74d3e0 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:50.671086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5f602-f3c2-42ef-9d96-ae7a6f74d3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:50.671086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5f602-f3c2-42ef-9d96-ae7a6f74d3e0 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:50.672432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 666/1338] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:50.677572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1916b-afc2-408b-9930-5a89e2d06f23 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:50.678021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1916b-afc2-408b-9930-5a89e2d06f23 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:50.678123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1916b-afc2-408b-9930-5a89e2d06f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:50.741501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-16b1916b-afc2-408b-9930-5a89e2d06f23 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:50.742696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-16b1916b-afc2-408b-9930-5a89e2d06f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:50.751177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:50.838541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 673/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:51.118311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5f602-f3c2-42ef-9d96-ae7a6f74d3e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:51.120460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1ca5f602-f3c2-42ef-9d96-ae7a6f74d3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:51.226712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5f602-f3c2-42ef-9d96-ae7a6f74d3e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:51.299929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:51.300533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:51.300807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:51.301104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:51.302726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:51.339221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:51.411133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:51.412530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836961a-65a7-4425-be4b-1e7c2922cfe3 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:51.430544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 667/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:36:55.523922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0581de8-6779-482d-9609-665b100fbbe0 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:55.524406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0581de8-6779-482d-9609-665b100fbbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:55.524514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0581de8-6779-482d-9609-665b100fbbe0 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:55.527554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 674/1341] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:55.532108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e855ba-46c9-44f2-8dd3-7d57b82e17c4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:55.532538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e855ba-46c9-44f2-8dd3-7d57b82e17c4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:55.532632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e855ba-46c9-44f2-8dd3-7d57b82e17c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:55.606552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c9e855ba-46c9-44f2-8dd3-7d57b82e17c4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:55.609091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c9e855ba-46c9-44f2-8dd3-7d57b82e17c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:55.704627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:55.719541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 668/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:55.998464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0581de8-6779-482d-9609-665b100fbbe0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:56.000006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b0581de8-6779-482d-9609-665b100fbbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:56.102461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0581de8-6779-482d-9609-665b100fbbe0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:56.140977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:56.184072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:56.184673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:56.184950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:56.185222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:56.186820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:56.296627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:56.298051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1161ce-c555-4b1b-8163-7f6a9a2d7f5b service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:56.317866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 675/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:36:59.106701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789441-bf86-4b6e-aeaf-6ce81d4c82ae None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:59.107251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789441-bf86-4b6e-aeaf-6ce81d4c82ae None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:59.107367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789441-bf86-4b6e-aeaf-6ce81d4c82ae None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:59.108736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 669/1344] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:36:59.114043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28cb88-04f4-418f-a346-795c38941d0d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:59.114445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28cb88-04f4-418f-a346-795c38941d0d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:59.114591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28cb88-04f4-418f-a346-795c38941d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:59.183383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-dd28cb88-04f4-418f-a346-795c38941d0d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:36:59.184654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-dd28cb88-04f4-418f-a346-795c38941d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:59.205186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:59.284590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 676/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:36:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:36:59.584825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f789441-bf86-4b6e-aeaf-6ce81d4c82ae None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:36:59.586344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4f789441-bf86-4b6e-aeaf-6ce81d4c82ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:59.697433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f789441-bf86-4b6e-aeaf-6ce81d4c82ae None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:36:59.770757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:36:59.771361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:36:59.771673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:36:59.771944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:36:59.773541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:36:59.819388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:36:59.889502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:36:59.890955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc8336-235d-4774-9e41-a9d8533fb2e0 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:36:59.909536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 670/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:36:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:04.007918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0aab-05a3-45ee-b8ac-8096625a9e87 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:04.008360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0aab-05a3-45ee-b8ac-8096625a9e87 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:04.008439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0aab-05a3-45ee-b8ac-8096625a9e87 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:04.010358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 677/1347] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:04.015921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e4f7c-4b92-4dbf-b9ce-c19adc0ff19e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:04.016360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e4f7c-4b92-4dbf-b9ce-c19adc0ff19e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:04.016602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e4f7c-4b92-4dbf-b9ce-c19adc0ff19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:04.080521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1d0e4f7c-4b92-4dbf-b9ce-c19adc0ff19e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:04.081718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1d0e4f7c-4b92-4dbf-b9ce-c19adc0ff19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:04.172978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:04.175302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 671/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:04.464527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0aab-05a3-45ee-b8ac-8096625a9e87 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:04.466049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-444c0aab-05a3-45ee-b8ac-8096625a9e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:04.569737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0aab-05a3-45ee-b8ac-8096625a9e87 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:04.620097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:04.645383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:04.645985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:04.646263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:04.646531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:04.648179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:04.756620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:04.758038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84a21b-b500-4df4-98cf-da5e5bd3f72a service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:04.775904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 678/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:09.731510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dc4ea-4dd7-4ea9-8d9e-f51133f00730 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:09.731944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dc4ea-4dd7-4ea9-8d9e-f51133f00730 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:09.732238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dc4ea-4dd7-4ea9-8d9e-f51133f00730 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:09.733697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 672/1350] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:09.742183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e02651-c90a-44e9-a6c8-19f08b3ca99b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:09.742183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e02651-c90a-44e9-a6c8-19f08b3ca99b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:09.742183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e02651-c90a-44e9-a6c8-19f08b3ca99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:09.898620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-46e02651-c90a-44e9-a6c8-19f08b3ca99b None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:09.900710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-46e02651-c90a-44e9-a6c8-19f08b3ca99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:09.921762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:10.024574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 679/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:37:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:14.122986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b983-9065-45e7-92d6-e0a30d9dc1b1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:14.123386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b983-9065-45e7-92d6-e0a30d9dc1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:14.123386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b983-9065-45e7-92d6-e0a30d9dc1b1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:14.124942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 673/1352] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:14.130375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09feb-827c-4650-b8f2-2bd56b5f6c7b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:14.130825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09feb-827c-4650-b8f2-2bd56b5f6c7b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:14.131136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09feb-827c-4650-b8f2-2bd56b5f6c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:14.212067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-98c09feb-827c-4650-b8f2-2bd56b5f6c7b None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:14.213443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-98c09feb-827c-4650-b8f2-2bd56b5f6c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:14.291564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:14.330800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 680/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:37:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:15.717311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add30f76-0ee9-47d1-b026-818fb1c68b55 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:15.717871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add30f76-0ee9-47d1-b026-818fb1c68b55 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:15.717871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add30f76-0ee9-47d1-b026-818fb1c68b55 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:15.719167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 674/1354] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:15.724196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5f00e-3c66-4b0f-af14-d993987ad2b6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:15.724616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5f00e-3c66-4b0f-af14-d993987ad2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:15.724801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5f00e-3c66-4b0f-af14-d993987ad2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:15.802771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-19b5f00e-3c66-4b0f-af14-d993987ad2b6 None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:15.804099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-19b5f00e-3c66-4b0f-af14-d993987ad2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:15.913446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 681/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:37:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:37:19.401886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9630a-9931-4624-bee8-805767594fdc None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:19.402422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9630a-9931-4624-bee8-805767594fdc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:19.402422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9630a-9931-4624-bee8-805767594fdc None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:19.403805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 675/1356] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:19.409336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f7e4-4118-4116-9d36-84d4131dc083 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:19.410128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f7e4-4118-4116-9d36-84d4131dc083 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:19.410561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f7e4-4118-4116-9d36-84d4131dc083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:19.448299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:19.484269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e527f7e4-4118-4116-9d36-84d4131dc083 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:19.485482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e527f7e4-4118-4116-9d36-84d4131dc083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:19.587574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 682/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:19.904309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911187da-d2c7-4550-bc52-13f4ddb417f5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:19.904875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911187da-d2c7-4550-bc52-13f4ddb417f5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:19.905364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911187da-d2c7-4550-bc52-13f4ddb417f5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:19.906930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 676/1358] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 00:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:19.928212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8613b-0ac3-4b21-92f4-1866525fb5f4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:19.928969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8613b-0ac3-4b21-92f4-1866525fb5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:19.929458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8613b-0ac3-4b21-92f4-1866525fb5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:20.018108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9ca8613b-0ac3-4b21-92f4-1866525fb5f4 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:20.019325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9ca8613b-0ac3-4b21-92f4-1866525fb5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:20.073273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:20.147538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 683/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:37:20.155607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911187da-d2c7-4550-bc52-13f4ddb417f5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:20.157116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-911187da-d2c7-4550-bc52-13f4ddb417f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:20.319905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911187da-d2c7-4550-bc52-13f4ddb417f5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:20.397328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:20.441494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:20.442183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:20.442617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:20.442935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:20.444894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:20.546895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:20.548766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17524-6926-4589-a781-6f3b27902fe3 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:20.568859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 677/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:22.950473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282a98a-88d8-4a57-b31a-64c7b21d1bc2 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:22.951053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282a98a-88d8-4a57-b31a-64c7b21d1bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:22.951053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282a98a-88d8-4a57-b31a-64c7b21d1bc2 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:22.956490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 684/1361] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:22.958207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e12b5-13a5-4acb-a84d-a4d5a44fe6c1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:22.958207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e12b5-13a5-4acb-a84d-a4d5a44fe6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:22.959080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e12b5-13a5-4acb-a84d-a4d5a44fe6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:23.031304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ae7e12b5-13a5-4acb-a84d-a4d5a44fe6c1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:23.032609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ae7e12b5-13a5-4acb-a84d-a4d5a44fe6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:23.077015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:23.142548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 678/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:23.432604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282a98a-88d8-4a57-b31a-64c7b21d1bc2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:23.434165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8282a98a-88d8-4a57-b31a-64c7b21d1bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:23.615307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282a98a-88d8-4a57-b31a-64c7b21d1bc2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:23.629048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:23.761692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:23.762292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:23.762565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:23.762830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:23.764457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:23.857244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:23.858866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab52113-72a9-46ad-80e9-335086884d3b service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:23.878646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 685/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:26.373397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6d91b-3999-4fdf-a280-0b679e4e7512 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:26.373942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6d91b-3999-4fdf-a280-0b679e4e7512 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:26.373942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6d91b-3999-4fdf-a280-0b679e4e7512 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:26.375279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 679/1364] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:37:26.381051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5d8d-a1ef-485c-9eb4-668781c6625a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:26.381506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5d8d-a1ef-485c-9eb4-668781c6625a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:26.381787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5d8d-a1ef-485c-9eb4-668781c6625a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:26.399141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:26.453085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7c5a5d8d-a1ef-485c-9eb4-668781c6625a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:26.454293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7c5a5d8d-a1ef-485c-9eb4-668781c6625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:26.555945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 686/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:26.851706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfa658-692a-43a5-8f4d-c97af750b276 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:26.852131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfa658-692a-43a5-8f4d-c97af750b276 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:26.852603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfa658-692a-43a5-8f4d-c97af750b276 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:26.854176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 680/1366] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 00:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:26.861006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714883ed-67af-4f6c-9865-038ee48dec34 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:26.861455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714883ed-67af-4f6c-9865-038ee48dec34 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:26.861819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714883ed-67af-4f6c-9865-038ee48dec34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:26.941684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-714883ed-67af-4f6c-9865-038ee48dec34 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:26.942952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-714883ed-67af-4f6c-9865-038ee48dec34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:26.980180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:27.066262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 687/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 00:37:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:27.074708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfa658-692a-43a5-8f4d-c97af750b276 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:27.076590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0ddfa658-692a-43a5-8f4d-c97af750b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:27.232510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfa658-692a-43a5-8f4d-c97af750b276 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:27.255167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:27.354984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:27.355783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:27.356175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:27.356560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:27.358268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:27.453196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:27.454964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872939b-5876-4f63-9164-dab9ff557161 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:27.473627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 681/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:27.624745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7071-8b4b-44ce-bfce-59969bbcaaf8 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:27.625280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7071-8b4b-44ce-bfce-59969bbcaaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:27.625280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7071-8b4b-44ce-bfce-59969bbcaaf8 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:27.626878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 688/1369] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:27.632859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f02cc-72a8-43ba-92b8-73bb9b475fb0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:27.633280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f02cc-72a8-43ba-92b8-73bb9b475fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:27.633791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f02cc-72a8-43ba-92b8-73bb9b475fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:27.680333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:27.722823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f32f02cc-72a8-43ba-92b8-73bb9b475fb0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:27.724354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f32f02cc-72a8-43ba-92b8-73bb9b475fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:27.835317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 682/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:37:28.166328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b159a8-d099-42c4-8186-517be3a02c25 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:28.166328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b159a8-d099-42c4-8186-517be3a02c25 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:28.166328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b159a8-d099-42c4-8186-517be3a02c25 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:28.168020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 689/1371] 10.222.0.15 () {54 vars in 964 bytes} [Thu Sep 15 00:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:28.172683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f2b4-6a36-4f03-8a5f-3be2a9c7bfd5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:28.172967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f2b4-6a36-4f03-8a5f-3be2a9c7bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:28.173234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f2b4-6a36-4f03-8a5f-3be2a9c7bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:28.253790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d848f2b4-6a36-4f03-8a5f-3be2a9c7bfd5 None None] MFA Rules not processed for user `7727b52cd17b41708b243dac7b4f2c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:28.253790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d848f2b4-6a36-4f03-8a5f-3be2a9c7bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:28.305592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:28.337983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 683/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Thu Sep 15 00:37:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:28.345685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b159a8-d099-42c4-8186-517be3a02c25 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:28.347278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-33b159a8-d099-42c4-8186-517be3a02c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:28.457236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b159a8-d099-42c4-8186-517be3a02c25 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:28.501612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:28.539769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:28.540360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:28.540759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:28.541129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:28.542873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:28.699469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:28.701844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5456d8-4bbd-4465-890c-e1f37de0c989 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:28.723263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 690/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:29.711026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e36eac-1491-4024-a939-ea5ef1532570 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:29.711593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e36eac-1491-4024-a939-ea5ef1532570 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:29.711593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e36eac-1491-4024-a939-ea5ef1532570 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:29.712942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 684/1374] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:29.718089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b78fc1-5cf4-4245-9894-b5dfc9368cf2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:29.718494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b78fc1-5cf4-4245-9894-b5dfc9368cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:29.718637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b78fc1-5cf4-4245-9894-b5dfc9368cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:29.784237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e3b78fc1-5cf4-4245-9894-b5dfc9368cf2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:29.785437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e3b78fc1-5cf4-4245-9894-b5dfc9368cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:29.806272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:29.883052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 691/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:30.171231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e36eac-1491-4024-a939-ea5ef1532570 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:30.172969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-66e36eac-1491-4024-a939-ea5ef1532570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:30.341402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e36eac-1491-4024-a939-ea5ef1532570 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:30.396494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:30.458214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:30.458816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:30.459087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:30.459356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:30.461502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:30.558818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:30.560487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044f1be-f3ea-48b6-891c-3a1c9a16097f service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:30.579315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 685/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:32.787150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce84e0-7919-4c06-b3bb-7d0e0df683bb None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:32.787750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce84e0-7919-4c06-b3bb-7d0e0df683bb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:32.787750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce84e0-7919-4c06-b3bb-7d0e0df683bb None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:32.789873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 692/1377] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:32.794999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26af210-45ca-4a89-bba6-e4275ee41a99 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:32.795417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26af210-45ca-4a89-bba6-e4275ee41a99 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:32.795631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26af210-45ca-4a89-bba6-e4275ee41a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:32.862788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b26af210-45ca-4a89-bba6-e4275ee41a99 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:32.863918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b26af210-45ca-4a89-bba6-e4275ee41a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:32.871813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:32.967522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 686/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:33.256040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce84e0-7919-4c06-b3bb-7d0e0df683bb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:33.257605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d7ce84e0-7919-4c06-b3bb-7d0e0df683bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:33.414439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce84e0-7919-4c06-b3bb-7d0e0df683bb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:33.418415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:33.531559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:33.532087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:33.532366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:33.532633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:33.534222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:33.627773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:33.629690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4137e-431b-4674-b0ce-0090561f42c5 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:33.652884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 693/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:35.847718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df420-2a1a-47fa-ad8e-4f1e4257de8b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:35.848246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df420-2a1a-47fa-ad8e-4f1e4257de8b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:35.848624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df420-2a1a-47fa-ad8e-4f1e4257de8b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:35.850124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 687/1380] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:35.855952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68bb3-cc63-4f35-a34e-75ed2b6fb704 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:35.856392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68bb3-cc63-4f35-a34e-75ed2b6fb704 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:35.856581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68bb3-cc63-4f35-a34e-75ed2b6fb704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:35.868446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:35.924881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-37a68bb3-cc63-4f35-a34e-75ed2b6fb704 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:35.926110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-37a68bb3-cc63-4f35-a34e-75ed2b6fb704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:36.031678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 694/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:36.320935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df420-2a1a-47fa-ad8e-4f1e4257de8b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:36.322462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce8df420-2a1a-47fa-ad8e-4f1e4257de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:36.463032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:36.473374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df420-2a1a-47fa-ad8e-4f1e4257de8b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:36.585849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:36.586455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:36.586733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:36.587007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:36.588642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:36.683386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:36.685040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677d2f2-9b69-4fe5-a4af-4617829ff52f service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:36.694927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:36.703706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 688/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:38.992112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99793da-3b78-46b8-95b5-56dc4bffb3c1 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:38.992632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99793da-3b78-46b8-95b5-56dc4bffb3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:38.992632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99793da-3b78-46b8-95b5-56dc4bffb3c1 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:38.993922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 695/1383] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:39.000004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8bd93-7032-4fb3-b8be-43d0b02d4d40 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:39.000400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8bd93-7032-4fb3-b8be-43d0b02d4d40 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:39.000715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8bd93-7032-4fb3-b8be-43d0b02d4d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:39.075028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-44f8bd93-7032-4fb3-b8be-43d0b02d4d40 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:39.076272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-44f8bd93-7032-4fb3-b8be-43d0b02d4d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:39.176363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 689/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:39.468125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99793da-3b78-46b8-95b5-56dc4bffb3c1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:39.469641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b99793da-3b78-46b8-95b5-56dc4bffb3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:39.555334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:39.620125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99793da-3b78-46b8-95b5-56dc4bffb3c1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:39.740264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:39.740875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:39.741712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:39.742003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:39.743636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:39.793907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:39.838351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:39.840111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f62dd8-82e9-4964-9e72-a1d648f89ab9 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:39.863589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 696/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:37:41.994975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d1d4a-b4bc-4d53-b275-551627115851 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:41.995727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d1d4a-b4bc-4d53-b275-551627115851 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:41.995727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d1d4a-b4bc-4d53-b275-551627115851 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:41.996803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 690/1386] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:42.002595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e4631-e80a-4891-a53d-ad3045ab7b4b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:42.003297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e4631-e80a-4891-a53d-ad3045ab7b4b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:42.003627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e4631-e80a-4891-a53d-ad3045ab7b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:42.083624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-196e4631-e80a-4891-a53d-ad3045ab7b4b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:42.085489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-196e4631-e80a-4891-a53d-ad3045ab7b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:42.190749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 697/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:42.541434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d1d4a-b4bc-4d53-b275-551627115851 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:42.542956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3a1d1d4a-b4bc-4d53-b275-551627115851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:42.644570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:42.731303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d1d4a-b4bc-4d53-b275-551627115851 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:42.850922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:42.851563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:42.851840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:42.852111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:42.853717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:42.892668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:42.952193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:42.954370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d9c40-9005-41b4-bcea-ca584f30904f service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:42.977657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 691/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:45.184482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb30848-dbfd-4bf4-84d7-432a05873511 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:45.185041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb30848-dbfd-4bf4-84d7-432a05873511 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:45.185041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb30848-dbfd-4bf4-84d7-432a05873511 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:45.186495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 698/1389] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:45.191847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2480ae-4444-4e55-8196-44ee252f59ae None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:45.192236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2480ae-4444-4e55-8196-44ee252f59ae None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:45.192395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2480ae-4444-4e55-8196-44ee252f59ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:45.267438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6a2480ae-4444-4e55-8196-44ee252f59ae None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:45.268669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6a2480ae-4444-4e55-8196-44ee252f59ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:45.367022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 692/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:45.664021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb30848-dbfd-4bf4-84d7-432a05873511 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:45.665563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2eb30848-dbfd-4bf4-84d7-432a05873511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:45.700982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:45.814700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb30848-dbfd-4bf4-84d7-432a05873511 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:45.920079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:45.932964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:45.933577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:45.933853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:45.934123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:45.935742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:46.026059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:46.027713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9a1e4-945f-46f3-89a6-982d1311e24b service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:46.045512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 699/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:46.384219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4528-ad9f-408b-aa95-964a39a772be None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:46.384806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4528-ad9f-408b-aa95-964a39a772be None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:46.384806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4528-ad9f-408b-aa95-964a39a772be None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:46.386115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 693/1392] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:46.391390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e09fe-d9a6-42a5-bfdd-7b62b694cb31 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:46.391879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e09fe-d9a6-42a5-bfdd-7b62b694cb31 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:46.392008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e09fe-d9a6-42a5-bfdd-7b62b694cb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:46.459262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-767e09fe-d9a6-42a5-bfdd-7b62b694cb31 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:46.460565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-767e09fe-d9a6-42a5-bfdd-7b62b694cb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:46.519573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:46.558812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 700/1393] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:46.858275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2312f-d094-493f-83e2-d9c5ce3465df None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:46.858559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2312f-d094-493f-83e2-d9c5ce3465df None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:46.858840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2312f-d094-493f-83e2-d9c5ce3465df None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:46.860233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 694/1394] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 00:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:46.865427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dcc89-2f89-4b03-afc1-d2b41038ba6f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:46.865709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dcc89-2f89-4b03-afc1-d2b41038ba6f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:46.865981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dcc89-2f89-4b03-afc1-d2b41038ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:46.942166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-382dcc89-2f89-4b03-afc1-d2b41038ba6f None None] MFA Rules not processed for user `0d2cf22448884e1f8758b9a448d001ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:46.943287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-382dcc89-2f89-4b03-afc1-d2b41038ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:47.022582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 701/1395] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 00:37:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:47.031558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2312f-d094-493f-83e2-d9c5ce3465df None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:47.033065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d0c2312f-d094-493f-83e2-d9c5ce3465df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:47.047828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:47.138177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2312f-d094-493f-83e2-d9c5ce3465df None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:47.213048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:47.213571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:47.213849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:47.214121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:47.215761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:47.284774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:47.328081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:47.329528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bab0b-79af-4c09-8a3c-96347b4abdb0 service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:47.347513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 695/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:48.089595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67c917-4778-49ae-aa6d-0bfe7c0cc41b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.090067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67c917-4778-49ae-aa6d-0bfe7c0cc41b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.090067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67c917-4778-49ae-aa6d-0bfe7c0cc41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.163989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-6a67c917-4778-49ae-aa6d-0bfe7c0cc41b None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:48.165246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6a67c917-4778-49ae-aa6d-0bfe7c0cc41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:48.174457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:48.250779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c51cd-22b2-40ba-a56a-7df939ffeb42 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.251337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c51cd-22b2-40ba-a56a-7df939ffeb42 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.251337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c51cd-22b2-40ba-a56a-7df939ffeb42 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.252672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 696/1397] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:37:48.256421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 702/1398] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:37:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:48.257911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.258129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.258401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.301892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319020c-258c-4f96-be04-6dfd9a24bb18 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.302255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319020c-258c-4f96-be04-6dfd9a24bb18 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.302591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319020c-258c-4f96-be04-6dfd9a24bb18 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.304164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 703/1399] 10.222.0.15 () {54 vars in 943 bytes} [Thu Sep 15 00:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:37:48.311177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.311652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.311995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.327218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:48.328466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:48.388113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] MFA Rules not processed for user `0587d0fb93ad478fa931141ac9e1a5de`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:48.389252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:48.424007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 697/1400] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:48.472137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:48.494928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 704/1401] 10.222.0.15 () {58 vars in 1040 bytes} [Thu Sep 15 00:37:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:48.504539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:48.505725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:48.532431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:48.610120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d77ff-a012-48c7-af02-c0baa0cfa290 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:48.714239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:48.714837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.715109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.715381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.717213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:48.722913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:48.724238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:48.759172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:48.801764 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:48.803393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64b876-097e-416b-b376-bc5e214dfde1 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:48.812225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 698/1402] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:37:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:37:48.821964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adacc1a-1cde-4578-843d-2b5b2ddb9c0e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:48.937322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:48.948607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:48.949334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:48.949615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:48.949880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:48.951661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:49.056495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:49.058356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bddeb-4da5-409b-896b-bde37955a972 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:49.080448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 705/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:37:51.303264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ea890-5ecb-49e4-93c4-366a81b6a4fb None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:51.303975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ea890-5ecb-49e4-93c4-366a81b6a4fb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:51.303975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ea890-5ecb-49e4-93c4-366a81b6a4fb None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:51.305245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 699/1404] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:51.311199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5a44-bf7a-4dec-9ef3-f02ac2496603 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:51.311697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5a44-bf7a-4dec-9ef3-f02ac2496603 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:51.312149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5a44-bf7a-4dec-9ef3-f02ac2496603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:51.381539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7baf5a44-bf7a-4dec-9ef3-f02ac2496603 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:51.382835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7baf5a44-bf7a-4dec-9ef3-f02ac2496603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:51.450167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:51.489513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 706/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:51.780590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ea890-5ecb-49e4-93c4-366a81b6a4fb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:51.782138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6a8ea890-5ecb-49e4-93c4-366a81b6a4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:51.942783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ea890-5ecb-49e4-93c4-366a81b6a4fb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:51.985880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:52.077892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:52.081352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:52.081662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:52.081964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:52.083896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:52.192290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:52.195264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500e97a-c849-4017-ae40-c0c63fa2f222 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:52.216701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 700/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:37:54.443434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5781232-5e36-433b-8228-19cce9fda963 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:54.444651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5781232-5e36-433b-8228-19cce9fda963 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:54.444651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5781232-5e36-433b-8228-19cce9fda963 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:54.446129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 707/1407] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:37:54.451828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232e6d-a8cc-48b2-b32c-2fefc83f349f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:54.452378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232e6d-a8cc-48b2-b32c-2fefc83f349f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:54.452378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232e6d-a8cc-48b2-b32c-2fefc83f349f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:54.529843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:54.558303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d4232e6d-a8cc-48b2-b32c-2fefc83f349f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:37:54.559584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d4232e6d-a8cc-48b2-b32c-2fefc83f349f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:54.679415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 701/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:37:54.979210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5781232-5e36-433b-8228-19cce9fda963 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:37:54.980235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d5781232-5e36-433b-8228-19cce9fda963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:55.145535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5781232-5e36-433b-8228-19cce9fda963 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:37:55.201887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:37:55.275609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:37:55.276145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:37:55.276539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:37:55.276813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:37:55.278443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:37:55.409748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:37:55.411386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d25e1-51c0-453a-a69d-2478bcedcd50 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:37:55.432423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 708/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:04.677267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=47 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:05.429872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=75 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:11.488671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce4fb0-3686-4916-9d9e-d94fc4bc2fbb None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:11.489349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce4fb0-3686-4916-9d9e-d94fc4bc2fbb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:11.489349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce4fb0-3686-4916-9d9e-d94fc4bc2fbb None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:11.490561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 702/1410] 10.222.0.15 () {54 vars in 918 bytes} [Thu Sep 15 00:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:11.496104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907988b-1f17-4388-8434-4af383468012 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:11.496530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907988b-1f17-4388-8434-4af383468012 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:11.496639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907988b-1f17-4388-8434-4af383468012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:11.571266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4907988b-1f17-4388-8434-4af383468012 None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:11.572511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4907988b-1f17-4388-8434-4af383468012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:11.675725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 709/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 15 00:38:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:11.690505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1e1b-5a19-4723-9aa5-128f21b5ae64 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:11.690809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1e1b-5a19-4723-9aa5-128f21b5ae64 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:11.691082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1e1b-5a19-4723-9aa5-128f21b5ae64 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:11.692555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 703/1412] 10.222.0.15 () {54 vars in 918 bytes} [Thu Sep 15 00:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:11.697777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccd73-7fe0-4296-9b53-f74626b70144 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:11.698094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccd73-7fe0-4296-9b53-f74626b70144 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:11.698368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccd73-7fe0-4296-9b53-f74626b70144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:11.771909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:11.774630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-3dfccd73-7fe0-4296-9b53-f74626b70144 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:11.776427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3dfccd73-7fe0-4296-9b53-f74626b70144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:11.888901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 710/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 15 00:38:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:14.448838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57a8e-0de8-40e3-9335-6fa43007fc78 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:14.449396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57a8e-0de8-40e3-9335-6fa43007fc78 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:14.449396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57a8e-0de8-40e3-9335-6fa43007fc78 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:14.450707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:14.455932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd06e-da72-4da8-8e5d-d6b9f784d423 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:14.456370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd06e-da72-4da8-8e5d-d6b9f784d423 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:14.456505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd06e-da72-4da8-8e5d-d6b9f784d423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:14.522293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2a7dd06e-da72-4da8-8e5d-d6b9f784d423 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:14.523527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2a7dd06e-da72-4da8-8e5d-d6b9f784d423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:14.577395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:14.618354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:14.907106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57a8e-0de8-40e3-9335-6fa43007fc78 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:14.908713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-41e57a8e-0de8-40e3-9335-6fa43007fc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:15.050756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57a8e-0de8-40e3-9335-6fa43007fc78 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:15.121558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:15.162034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:15.162592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:15.162869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:15.163137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:15.164749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:15.252226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:15.253833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a86dfc-66fd-4eb4-8135-6f829918a93e service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:15.272821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 705/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:16.411689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6bf14-680e-410f-9806-6fcfaadf2a95 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:16.412246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6bf14-680e-410f-9806-6fcfaadf2a95 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:16.412246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6bf14-680e-410f-9806-6fcfaadf2a95 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:16.413586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:16.419756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fe81d-74d7-4474-a915-001a97675e84 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:16.420202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fe81d-74d7-4474-a915-001a97675e84 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:16.420309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fe81d-74d7-4474-a915-001a97675e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:16.494920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-801fe81d-74d7-4474-a915-001a97675e84 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:16.496162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-801fe81d-74d7-4474-a915-001a97675e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:16.523631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:16.591852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:16.747785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6bf14-680e-410f-9806-6fcfaadf2a95 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:16.749290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-37a6bf14-680e-410f-9806-6fcfaadf2a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:16.893534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6bf14-680e-410f-9806-6fcfaadf2a95 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:16.932525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:17.010719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:17.011323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:17.011643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:17.011918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:17.013533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:17.102074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:17.103690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5472d9-5b6a-435e-8fbc-1c53689b0b5d service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:17.121808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 713/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:18.258035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186bb9a-d6d4-431d-ac8f-340bcd53b3d7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:18.258880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186bb9a-d6d4-431d-ac8f-340bcd53b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:18.258880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186bb9a-d6d4-431d-ac8f-340bcd53b3d7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:18.260647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:18.266717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b45d4-64ba-4351-9e40-66e4cce6618a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:18.267150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b45d4-64ba-4351-9e40-66e4cce6618a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:18.267268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b45d4-64ba-4351-9e40-66e4cce6618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:18.321939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:18.344101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-db4b45d4-64ba-4351-9e40-66e4cce6618a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:18.345329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-db4b45d4-64ba-4351-9e40-66e4cce6618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:18.437114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:38:18.587452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186bb9a-d6d4-431d-ac8f-340bcd53b3d7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:18.588979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7186bb9a-d6d4-431d-ac8f-340bcd53b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:18.737657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186bb9a-d6d4-431d-ac8f-340bcd53b3d7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:18.749445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:18.848383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:18.848907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:18.849180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:18.849444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:18.851137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:18.939779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:18.941380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41bbb6-e061-4b3f-bb3c-fc0793108245 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:18.959190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 708/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:20.093405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d736d72-20e0-442d-aca6-f5b06475aaa7 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:20.093965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d736d72-20e0-442d-aca6-f5b06475aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:20.093965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d736d72-20e0-442d-aca6-f5b06475aaa7 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:20.095298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:20.101510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa689f-8f6a-4ac4-8348-3169e269e4ac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:20.101930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa689f-8f6a-4ac4-8348-3169e269e4ac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:20.102044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa689f-8f6a-4ac4-8348-3169e269e4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:20.121301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:20.176606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-90fa689f-8f6a-4ac4-8348-3169e269e4ac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:20.177802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-90fa689f-8f6a-4ac4-8348-3169e269e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:20.273503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:20.423275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d736d72-20e0-442d-aca6-f5b06475aaa7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:20.424847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6d736d72-20e0-442d-aca6-f5b06475aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:20.533111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:20.568429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d736d72-20e0-442d-aca6-f5b06475aaa7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:20.680692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:20.681301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:20.681601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:20.681871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:20.683513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:20.760516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:20.774193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:20.775832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cf2bb-4437-4143-aa4a-db51fe0cda0d service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:20.793694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 716/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:21.899143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43373-b4f5-4d74-9f7e-182b63dc571a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:21.899706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43373-b4f5-4d74-9f7e-182b63dc571a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:21.899835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43373-b4f5-4d74-9f7e-182b63dc571a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:21.901339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:38:21.907622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e600-e1e1-4e85-9aea-703ac1910680 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:21.908048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e600-e1e1-4e85-9aea-703ac1910680 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:21.908156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e600-e1e1-4e85-9aea-703ac1910680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:21.982657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5d19e600-e1e1-4e85-9aea-703ac1910680 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:21.983811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5d19e600-e1e1-4e85-9aea-703ac1910680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:22.078059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:22.226363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43373-b4f5-4d74-9f7e-182b63dc571a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:22.227907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fe43373-b4f5-4d74-9f7e-182b63dc571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:22.313254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:22.373264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43373-b4f5-4d74-9f7e-182b63dc571a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:22.481855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:22.482466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:22.482743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:22.483007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:22.484699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:22.535736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:22.576523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:22.578111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab04d9-64e6-4681-b62b-fe70c21f0d59 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:22.598222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 711/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:23.648858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b88e-e3af-4536-9be0-dc4d8d30fc7a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:23.649414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b88e-e3af-4536-9be0-dc4d8d30fc7a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:23.649414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b88e-e3af-4536-9be0-dc4d8d30fc7a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:23.650729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:23.657161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343393c-90a9-4c2c-a50e-0d28a92c10d0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:23.657611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343393c-90a9-4c2c-a50e-0d28a92c10d0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:23.657727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343393c-90a9-4c2c-a50e-0d28a92c10d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:23.734380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b343393c-90a9-4c2c-a50e-0d28a92c10d0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:23.735653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b343393c-90a9-4c2c-a50e-0d28a92c10d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:23.830577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:23.984222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b88e-e3af-4536-9be0-dc4d8d30fc7a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:23.985655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7a51b88e-e3af-4536-9be0-dc4d8d30fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:24.034241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:24.128549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b88e-e3af-4536-9be0-dc4d8d30fc7a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:24.237270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:24.237787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:24.238060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:24.238325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:24.240008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:24.246993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:24.449614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:24.451176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61ef8f-7bfe-4299-989b-3ea662e2b56f service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:24.469093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 719/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:25.516305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e697505-5780-4275-b75e-c4570dfd1433 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:25.516904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e697505-5780-4275-b75e-c4570dfd1433 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:25.516904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e697505-5780-4275-b75e-c4570dfd1433 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:25.518200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:25.525573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a9b3-3144-413d-ae7e-00563eb2b298 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:25.526112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a9b3-3144-413d-ae7e-00563eb2b298 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:25.526112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a9b3-3144-413d-ae7e-00563eb2b298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:25.610238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8869a9b3-3144-413d-ae7e-00563eb2b298 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:25.611454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8869a9b3-3144-413d-ae7e-00563eb2b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:25.679115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:25.707608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:25.857798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e697505-5780-4275-b75e-c4570dfd1433 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:25.859299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8e697505-5780-4275-b75e-c4570dfd1433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:25.877030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:26.002762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e697505-5780-4275-b75e-c4570dfd1433 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:26.089019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:26.115117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:26.116290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:26.116569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:26.116840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:26.118424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:26.209485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:26.211090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fc5d7-61b9-4e65-883c-d67e46151782 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:26.229125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 714/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:27.279189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a96927-1efd-45a3-ad22-d7eb2407e1cd None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:27.279814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a96927-1efd-45a3-ad22-d7eb2407e1cd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:27.279814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a96927-1efd-45a3-ad22-d7eb2407e1cd None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:27.281135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:27.288065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eece7c-93e7-45de-b965-430657497ad4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:27.288532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eece7c-93e7-45de-b965-430657497ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:27.288639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eece7c-93e7-45de-b965-430657497ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:27.363128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-93eece7c-93e7-45de-b965-430657497ad4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:27.364342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-93eece7c-93e7-45de-b965-430657497ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:27.405594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:27.460238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:27.611829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a96927-1efd-45a3-ad22-d7eb2407e1cd None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:27.613385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d7a96927-1efd-45a3-ad22-d7eb2407e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:27.757621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a96927-1efd-45a3-ad22-d7eb2407e1cd None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:27.812129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:27.870438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:27.870947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:27.871217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:27.871502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:27.873081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:27.958772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:27.960421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c18f6-ce79-4ec3-b941-38d6b958c1fc service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:27.977540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 722/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:29.026889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1cd40-35ad-4787-ad08-f47799598c65 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:29.027403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1cd40-35ad-4787-ad08-f47799598c65 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:29.027403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1cd40-35ad-4787-ad08-f47799598c65 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:29.028733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:29.035083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba397d-613a-4ec0-aa35-1a7e24d7d8c9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:29.035563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba397d-613a-4ec0-aa35-1a7e24d7d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:29.035669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba397d-613a-4ec0-aa35-1a7e24d7d8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:29.110264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2dba397d-613a-4ec0-aa35-1a7e24d7d8c9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:29.111463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2dba397d-613a-4ec0-aa35-1a7e24d7d8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:29.113481 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:29.211683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:29.363059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1cd40-35ad-4787-ad08-f47799598c65 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:29.364605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3ef1cd40-35ad-4787-ad08-f47799598c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:29.506990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1cd40-35ad-4787-ad08-f47799598c65 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:29.533187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:29.618666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:29.619182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:29.619455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:29.619748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:29.621329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:29.709891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:29.711669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cf832-cc98-43c2-b320-6ab541acb35b service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:29.729397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 717/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:38:31.783555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d89d7-d92d-4ae0-9e07-4b624d817354 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:31.784129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d89d7-d92d-4ae0-9e07-4b624d817354 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:31.784129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d89d7-d92d-4ae0-9e07-4b624d817354 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:31.785464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:31.791503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c292f94-9227-4f8e-a98f-e127b0540652 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:31.791903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c292f94-9227-4f8e-a98f-e127b0540652 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:31.792051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c292f94-9227-4f8e-a98f-e127b0540652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:31.831384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:31.859267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5c292f94-9227-4f8e-a98f-e127b0540652 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:31.860496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5c292f94-9227-4f8e-a98f-e127b0540652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:31.954251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:32.244216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d89d7-d92d-4ae0-9e07-4b624d817354 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:32.245724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-983d89d7-d92d-4ae0-9e07-4b624d817354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:32.352313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d89d7-d92d-4ae0-9e07-4b624d817354 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:32.384468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:32.424978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:32.425580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:32.425854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:32.426122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:32.427748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:32.534234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:32.535607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b009db-78b0-4019-84ae-96145f83d8a6 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:32.552766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 725/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:34.850470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4a0de-9924-4046-9059-49e47c31d727 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:34.851023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4a0de-9924-4046-9059-49e47c31d727 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:34.851023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4a0de-9924-4046-9059-49e47c31d727 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:34.852395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:34.857677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e58cb2-705e-415c-be3c-212d0f613bf0 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:34.858110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e58cb2-705e-415c-be3c-212d0f613bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:34.858219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e58cb2-705e-415c-be3c-212d0f613bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:34.926068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b4e58cb2-705e-415c-be3c-212d0f613bf0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:34.927208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b4e58cb2-705e-415c-be3c-212d0f613bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:34.940419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:35.023457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:35.304303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4a0de-9924-4046-9059-49e47c31d727 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:35.305857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-29e4a0de-9924-4046-9059-49e47c31d727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:35.409516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4a0de-9924-4046-9059-49e47c31d727 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:35.422670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:35.486670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:35.487270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:35.487574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:35.487847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:35.489443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:35.598927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:35.601009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360c5f7-c740-49e3-8bee-7891559721fa service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:35.618863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 720/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:38.053604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed074f-4966-42d4-9fc3-444b2168845e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:38.054159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed074f-4966-42d4-9fc3-444b2168845e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:38.054159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed074f-4966-42d4-9fc3-444b2168845e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:38.055438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:38.060804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f501-c425-4638-91a8-31b0e2f8cac0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:38.061211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f501-c425-4638-91a8-31b0e2f8cac0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:38.061350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f501-c425-4638-91a8-31b0e2f8cac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:38.108569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:38.126859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ac60f501-c425-4638-91a8-31b0e2f8cac0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:38.128083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ac60f501-c425-4638-91a8-31b0e2f8cac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:38.222847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:38.511014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed074f-4966-42d4-9fc3-444b2168845e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:38.512606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0fed074f-4966-42d4-9fc3-444b2168845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:38.613148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed074f-4966-42d4-9fc3-444b2168845e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:38.653874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:38.687979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:38.688608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:38.688884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:38.689153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:38.690740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:38.799408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:38.800769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68957-5341-4b94-921c-15edc262d18c service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:38.819005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 728/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:41.089697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad609ec9-6f9c-47f6-93df-702795053a0b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:41.090247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad609ec9-6f9c-47f6-93df-702795053a0b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:41.090247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad609ec9-6f9c-47f6-93df-702795053a0b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:41.091695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:41.097034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc21b-5eaf-4d58-8d0d-8077fa7233c3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:41.097445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc21b-5eaf-4d58-8d0d-8077fa7233c3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:41.097562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc21b-5eaf-4d58-8d0d-8077fa7233c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:41.161883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-73ddc21b-5eaf-4d58-8d0d-8077fa7233c3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:41.163122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-73ddc21b-5eaf-4d58-8d0d-8077fa7233c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:41.182710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:41.257667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:41.545143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad609ec9-6f9c-47f6-93df-702795053a0b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:41.546676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ad609ec9-6f9c-47f6-93df-702795053a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:41.649826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad609ec9-6f9c-47f6-93df-702795053a0b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:41.672890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:41.727319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:41.727952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:41.728230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:41.728499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:41.730095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:41.837917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:41.839351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9deb4-682b-4c41-9a1f-b2b08e101342 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:41.857198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 723/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:44.073903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8073134-0fc7-4426-9da1-f67a3a53f38e None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:44.074434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8073134-0fc7-4426-9da1-f67a3a53f38e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:44.074434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8073134-0fc7-4426-9da1-f67a3a53f38e None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:44.075825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:44.081010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff819c-04ac-46a5-861e-feecdf903f56 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:44.081439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff819c-04ac-46a5-861e-feecdf903f56 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:44.081561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff819c-04ac-46a5-861e-feecdf903f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:44.145435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:44.148309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-abff819c-04ac-46a5-861e-feecdf903f56 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:44.150046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-abff819c-04ac-46a5-861e-feecdf903f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:44.246010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:38:44.528660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8073134-0fc7-4426-9da1-f67a3a53f38e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:44.530177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e8073134-0fc7-4426-9da1-f67a3a53f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:44.631188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8073134-0fc7-4426-9da1-f67a3a53f38e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:44.680543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:44.705549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:44.706314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:44.706852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:44.707329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:44.709712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:44.821324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:44.822656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5da58-7e78-4b4c-bfb9-e9d8444ce7d4 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:44.841106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 731/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:47.033791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add1a3-d0d0-46b6-ac89-d115d55eb2f5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:47.034326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add1a3-d0d0-46b6-ac89-d115d55eb2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:47.034326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add1a3-d0d0-46b6-ac89-d115d55eb2f5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:47.035625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:47.041602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20d2c-0bfd-4ee7-96a9-3e2ab1727618 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:47.041983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20d2c-0bfd-4ee7-96a9-3e2ab1727618 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:47.042175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20d2c-0bfd-4ee7-96a9-3e2ab1727618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:47.107507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8bb20d2c-0bfd-4ee7-96a9-3e2ab1727618 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:47.108747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8bb20d2c-0bfd-4ee7-96a9-3e2ab1727618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:47.138382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:47.204495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:47.492797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add1a3-d0d0-46b6-ac89-d115d55eb2f5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:47.494320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-82add1a3-d0d0-46b6-ac89-d115d55eb2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:47.595691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add1a3-d0d0-46b6-ac89-d115d55eb2f5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:47.624237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:47.670551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:47.671160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:47.671435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:47.671734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:47.673344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:47.784146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:47.785517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f31d69-7762-4d61-ab86-ebcbde6616d7 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:47.804412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 726/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:50.107041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9720b-54a6-4821-9c6f-e6668b730d1b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:50.107621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9720b-54a6-4821-9c6f-e6668b730d1b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:50.107621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9720b-54a6-4821-9c6f-e6668b730d1b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:50.108969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:50.114256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32b118-8b6f-4bf1-aede-2add8ae79546 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:50.114688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32b118-8b6f-4bf1-aede-2add8ae79546 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:50.114799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32b118-8b6f-4bf1-aede-2add8ae79546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:50.180546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0d32b118-8b6f-4bf1-aede-2add8ae79546 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:50.181754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0d32b118-8b6f-4bf1-aede-2add8ae79546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:50.189896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:50.277020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:50.562088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9720b-54a6-4821-9c6f-e6668b730d1b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:50.563677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-62e9720b-54a6-4821-9c6f-e6668b730d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:50.664406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9720b-54a6-4821-9c6f-e6668b730d1b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:50.718893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:50.737825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:50.738425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:50.738703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:50.738974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:50.740658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:50.847171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:50.848616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4bf9a-94c4-43ab-bafc-171c4b2925ad service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:50.865779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 734/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:53.059763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658e4a-483d-45a9-a5ee-75fad7e1c0c8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:53.060270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658e4a-483d-45a9-a5ee-75fad7e1c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:53.060270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658e4a-483d-45a9-a5ee-75fad7e1c0c8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:53.061567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:53.066787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4f23b-03b2-4ef3-8b8e-90d8efd302f5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:53.067182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4f23b-03b2-4ef3-8b8e-90d8efd302f5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:53.067388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4f23b-03b2-4ef3-8b8e-90d8efd302f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:53.131571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-48d4f23b-03b2-4ef3-8b8e-90d8efd302f5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:53.132760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-48d4f23b-03b2-4ef3-8b8e-90d8efd302f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:53.171156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:53.229061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:53.514935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42658e4a-483d-45a9-a5ee-75fad7e1c0c8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:53.516507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-42658e4a-483d-45a9-a5ee-75fad7e1c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:53.618094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42658e4a-483d-45a9-a5ee-75fad7e1c0c8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:53.656306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:53.693332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:53.693937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:53.694212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:53.694508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:53.696329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:53.806135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:53.807513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77e163-4f7e-4dab-8352-07b2df590ff0 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:53.942413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 729/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:56.121556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcfbba-4365-42c3-b1f0-b5e4f8416d7b None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:56.122104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcfbba-4365-42c3-b1f0-b5e4f8416d7b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:56.122104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcfbba-4365-42c3-b1f0-b5e4f8416d7b None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:56.123469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:56.128701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bed3-73c3-4e16-850a-807cf362f346 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:56.129138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bed3-73c3-4e16-850a-807cf362f346 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:56.129237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bed3-73c3-4e16-850a-807cf362f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:56.193954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c389bed3-73c3-4e16-850a-807cf362f346 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:56.195129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c389bed3-73c3-4e16-850a-807cf362f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:56.213048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:56.294283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:56.582856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcfbba-4365-42c3-b1f0-b5e4f8416d7b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:56.584472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-88dcfbba-4365-42c3-b1f0-b5e4f8416d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:56.685443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcfbba-4365-42c3-b1f0-b5e4f8416d7b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:56.750648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:56.758401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:56.759010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:56.759286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:56.759584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:56.761216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:56.866942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:56.868305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98680573-121f-4aa4-96c0-9acbbc0f3f67 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:56.886379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 737/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:38:59.034927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b3724-a8ad-4b7b-a55d-0ba7e03dfeb5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:59.035542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b3724-a8ad-4b7b-a55d-0ba7e03dfeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:59.035542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b3724-a8ad-4b7b-a55d-0ba7e03dfeb5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:59.036829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:38:59.042642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027658f-9dff-4859-a528-988899a0892e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:59.043078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027658f-9dff-4859-a528-988899a0892e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:59.043178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027658f-9dff-4859-a528-988899a0892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:59.108788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4027658f-9dff-4859-a528-988899a0892e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:38:59.109988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4027658f-9dff-4859-a528-988899a0892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:59.156919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:59.203458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:38:59.486518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b3724-a8ad-4b7b-a55d-0ba7e03dfeb5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:38:59.488122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d05b3724-a8ad-4b7b-a55d-0ba7e03dfeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:59.590027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b3724-a8ad-4b7b-a55d-0ba7e03dfeb5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:38:59.637060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:38:59.663604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:38:59.664203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:38:59.664479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:38:59.664748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:38:59.666334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:38:59.777119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:38:59.778485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5f928-0308-4a9b-8eed-141255f62c0c service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:38:59.797023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 732/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:09.202002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=22 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:09.794763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=66 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:19.768815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070baa0-428e-4608-949b-740fd466caa2 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:19.769355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070baa0-428e-4608-949b-740fd466caa2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:19.769355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070baa0-428e-4608-949b-740fd466caa2 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:19.770786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 739/1471] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:39:19.776392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20dd5-a0a9-4ea7-9c02-a1766348d1c5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:19.776790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20dd5-a0a9-4ea7-9c02-a1766348d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:19.776931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20dd5-a0a9-4ea7-9c02-a1766348d1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:19.840354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-65f20dd5-a0a9-4ea7-9c02-a1766348d1c5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:19.841533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-65f20dd5-a0a9-4ea7-9c02-a1766348d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:19.935630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 733/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:20.231334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d0d9e-a4bc-498c-a118-941d602fef20 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:20.231662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d0d9e-a4bc-498c-a118-941d602fef20 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:20.231969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d0d9e-a4bc-498c-a118-941d602fef20 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:20.233637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 740/1473] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 00:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:20.238606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499ad2-cbd9-415a-8695-d24d2bb77b02 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:20.238885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499ad2-cbd9-415a-8695-d24d2bb77b02 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:20.239154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499ad2-cbd9-415a-8695-d24d2bb77b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:20.314127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-61499ad2-cbd9-415a-8695-d24d2bb77b02 None None] MFA Rules not processed for user `0d2cf22448884e1f8758b9a448d001ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:20.315319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-61499ad2-cbd9-415a-8695-d24d2bb77b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:20.335893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:20.390905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 734/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 00:39:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:20.399165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d0d9e-a4bc-498c-a118-941d602fef20 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:20.400719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-dc2d0d9e-a4bc-498c-a118-941d602fef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:20.502370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d0d9e-a4bc-498c-a118-941d602fef20 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:20.573424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:20.574015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:20.574287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:20.574550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:20.576149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:20.628626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:20.683882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:20.685294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436a445-12f6-4f94-953a-e92486ef4aff service glance] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:20.703134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 741/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:22.922888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75e808-680a-4ba1-81b2-ddfb0882b4b5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:22.923411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75e808-680a-4ba1-81b2-ddfb0882b4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:22.923411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75e808-680a-4ba1-81b2-ddfb0882b4b5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:22.924723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 735/1476] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:22.929985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ef630-b3dc-4972-abdb-56091e1be691 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:22.930432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ef630-b3dc-4972-abdb-56091e1be691 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:22.930555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ef630-b3dc-4972-abdb-56091e1be691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:22.996343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e41ef630-b3dc-4972-abdb-56091e1be691 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:22.997548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e41ef630-b3dc-4972-abdb-56091e1be691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:23.092769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 742/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:23.371216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75e808-680a-4ba1-81b2-ddfb0882b4b5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:23.372807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4a75e808-680a-4ba1-81b2-ddfb0882b4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:23.502197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75e808-680a-4ba1-81b2-ddfb0882b4b5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:23.550430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:23.593836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:23.594425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:23.594693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:23.594956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:23.595613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:23.597309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:39:23.597309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:39:23.597309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:39:23.598476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b12973-1250-45cf-b8bf-6cf93f93d666 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:23.617277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 736/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 00:39:23 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:39:25.779040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c1197-70d9-45d8-b0c9-4e81ea3553f8 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:25.779584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c1197-70d9-45d8-b0c9-4e81ea3553f8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:25.779706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c1197-70d9-45d8-b0c9-4e81ea3553f8 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:25.780969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 743/1479] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:25.786219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1703a7b-08d3-4c1b-9bc8-1b7abc9b93f0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:25.786680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1703a7b-08d3-4c1b-9bc8-1b7abc9b93f0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:25.786790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1703a7b-08d3-4c1b-9bc8-1b7abc9b93f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:25.851641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d1703a7b-08d3-4c1b-9bc8-1b7abc9b93f0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:25.852892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d1703a7b-08d3-4c1b-9bc8-1b7abc9b93f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:25.951008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 737/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:26.246818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c1197-70d9-45d8-b0c9-4e81ea3553f8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:26.248383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-107c1197-70d9-45d8-b0c9-4e81ea3553f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:26.258509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:26.392663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c1197-70d9-45d8-b0c9-4e81ea3553f8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:26.471809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:26.504455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:26.504994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:26.505269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:26.505791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:26.507665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:26.594638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:26.596249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9b949-061a-4a6c-b6bb-b4fb6d43fd46 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:26.614629 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 744/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:28.921724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25f92-e4e6-4d57-ae14-d05a2f1e0773 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:28.922311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25f92-e4e6-4d57-ae14-d05a2f1e0773 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:28.922311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25f92-e4e6-4d57-ae14-d05a2f1e0773 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:28.923712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 738/1482] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:28.929525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907456de-05f4-4e42-a09d-0764568a2eab None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:28.929949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907456de-05f4-4e42-a09d-0764568a2eab None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:28.930063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907456de-05f4-4e42-a09d-0764568a2eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:28.995321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-907456de-05f4-4e42-a09d-0764568a2eab None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:28.996667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-907456de-05f4-4e42-a09d-0764568a2eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:29.043866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:29.093070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 745/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:29.385001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca25f92-e4e6-4d57-ae14-d05a2f1e0773 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:29.386571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7ca25f92-e4e6-4d57-ae14-d05a2f1e0773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:29.410672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:29.489497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca25f92-e4e6-4d57-ae14-d05a2f1e0773 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:29.561269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:29.561858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:29.562131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:29.562396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:29.564003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:29.637519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:29.673088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:29.674526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fab848-ea4f-440e-bd69-3306cca78a39 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:29.692263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 739/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:31.884143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab3e0a-8b16-473d-b3b5-db8635f008d5 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:31.884711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab3e0a-8b16-473d-b3b5-db8635f008d5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:31.884711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab3e0a-8b16-473d-b3b5-db8635f008d5 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:31.885990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 746/1485] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:31.891154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b404677-457b-4c92-b0eb-0b472dc8cdec None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:31.891561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b404677-457b-4c92-b0eb-0b472dc8cdec None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:31.891739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b404677-457b-4c92-b0eb-0b472dc8cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:31.956231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6b404677-457b-4c92-b0eb-0b472dc8cdec None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:31.957342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6b404677-457b-4c92-b0eb-0b472dc8cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:32.050870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 740/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:32.337969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab3e0a-8b16-473d-b3b5-db8635f008d5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:32.339513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fcab3e0a-8b16-473d-b3b5-db8635f008d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:32.441342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab3e0a-8b16-473d-b3b5-db8635f008d5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:32.510873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:32.512357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:32.513003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:32.513464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:32.513888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:32.515666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:32.622198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:32.623666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d022d-6668-4ea0-91e5-f1c8b950fe2e service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:32.641570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 747/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:34.883086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b2e5c-7946-4720-aeae-1e541e9b7eda None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:34.883728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b2e5c-7946-4720-aeae-1e541e9b7eda None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:34.883728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b2e5c-7946-4720-aeae-1e541e9b7eda None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:34.884980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 741/1488] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:34.890338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b82216-e178-4235-b553-bbd6ed341eb3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:34.890751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b82216-e178-4235-b553-bbd6ed341eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:34.890887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b82216-e178-4235-b553-bbd6ed341eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:34.956840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-65b82216-e178-4235-b553-bbd6ed341eb3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:34.957961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-65b82216-e178-4235-b553-bbd6ed341eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:35.009146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:35.052716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 748/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:35.347160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b2e5c-7946-4720-aeae-1e541e9b7eda None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:35.348736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ee6b2e5c-7946-4720-aeae-1e541e9b7eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:35.385207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:35.452208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b2e5c-7946-4720-aeae-1e541e9b7eda None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:35.524313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:35.524916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:35.525192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:35.525456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:35.527037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:35.610735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:35.639331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:35.640836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416d911-a43f-4ae9-af32-c7bb66822b9a service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:35.659068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 742/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:37.968723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b2c74-5c55-41b7-9817-b186552cb77a None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:37.969250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b2c74-5c55-41b7-9817-b186552cb77a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:37.969250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b2c74-5c55-41b7-9817-b186552cb77a None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:37.970521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 749/1491] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 00:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:39:37.976455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcf473-1f15-4093-80a2-29f2ea15eebe None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:37.976884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcf473-1f15-4093-80a2-29f2ea15eebe None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:37.976991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcf473-1f15-4093-80a2-29f2ea15eebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:38.042913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d2dcf473-1f15-4093-80a2-29f2ea15eebe None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:39:38.044133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d2dcf473-1f15-4093-80a2-29f2ea15eebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:38.137796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 743/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 00:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:39:38.425755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b2c74-5c55-41b7-9817-b186552cb77a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:39:38.427294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cc5b2c74-5c55-41b7-9817-b186552cb77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:38.528208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b2c74-5c55-41b7-9817-b186552cb77a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:39:38.599180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:39:38.599835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:39:38.600112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:39:38.600376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:39:38.601996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:39:38.610236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:38.712008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:39:38.713442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383042c-c719-45b7-9dae-f1735c47a7cb service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:39:38.731024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 750/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:39:48.136294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=82 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:39:48.728767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=50 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:58.018851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:58.019412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:58.019412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:58.093167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:40:58.094361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:58.187909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 744/1494] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:40:58.196939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:58.197445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:58.197544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:58.276339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:40:58.277453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:58.371261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 751/1495] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:40:58.375435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:58.376655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:58.388242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c802c35-bf8e-48cd-932f-f7b3d7ae2711 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:58.448820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:58.482935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:58.483562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:58.483840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:58.484111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:58.484756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:40:58.485055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:58.486073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:40:58.486073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:40:58.486073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:40:58.486958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:58.498923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 745/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:40:58.507723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:58.517104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6317793-f2ce-46c8-b5f1-22a12f6e7e37 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:58.576749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:58.610578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:58.611170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:58.611443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:58.611739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:58.612373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:58.614077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:40:58.614077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:40:58.614077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:40:58.615316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:58.671067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 752/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:40:58 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:40:58.675327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:58.676961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:58.709821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1625-8978-4892-92cc-39bbdd90ab20 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:58.801171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:58.801768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:58.802067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:58.802330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:58.802943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:58.803964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:40:58.803964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:40:58.803964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:40:58.805159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:58.845818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 746/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:40:58 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:40:58.850670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:58.852408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:58.914055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9664e4-4261-4547-916b-5a558ebaa6d1 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:58.995156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:59.006766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:59.007359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.007659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.007921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.008556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:59.009548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:40:59.009548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:40:59.009548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:40:59.010427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:59.022401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 753/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:40:59.026505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:59.028012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:59.039566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7798-bcb1-48e1-ab89-e86f085d0c77 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:59.065488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:59.134060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:59.134577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.134848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.135110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.135767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:59.136762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:40:59.136762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:40:59.136762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:40:59.137646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea14a4-9eb3-41d8-8a6c-38d016d256a6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:59.149540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 747/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:40:59.153843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:59.155335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:59.188562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49fb44-81df-47d8-8c0a-6f6ce2aad237 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:59.281282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:59.281976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.282197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.282468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] PATH_INFO: `/v3/projects/3baf00ec8bbb40ee87f6ce698f331fed/users/7b0c7516e6374410a65616ad3f9193f9/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.305446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3baf00ec8bbb40ee87f6ce698f331fed, user_id=7b0c7516e6374410a65616ad3f9193f9, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:59.306460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:40:59.306460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:40:59.306460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:40:59.308409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed896a3-b798-4140-9c36-139d34bd0483 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:59.331887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 754/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:40:59 2022] PUT /identity/v3/projects/3baf00ec8bbb40ee87f6ce698f331fed/users/7b0c7516e6374410a65616ad3f9193f9/roles/87f1831257824560813f954348534540 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:40:59.344461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e106ec8-3730-4a14-bf78-d09c2a668e9e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.344819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e106ec8-3730-4a14-bf78-d09c2a668e9e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.345130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e106ec8-3730-4a14-bf78-d09c2a668e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.401103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8e106ec8-3730-4a14-bf78-d09c2a668e9e None None] MFA Rules not processed for user `7b0c7516e6374410a65616ad3f9193f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:40:59.402228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8e106ec8-3730-4a14-bf78-d09c2a668e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:59.477329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 748/1502] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:40:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:40:59.490477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1d18c-5984-4cb8-88d4-916021790ea7 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.490761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1d18c-5984-4cb8-88d4-916021790ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.491030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1d18c-5984-4cb8-88d4-916021790ea7 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.492382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 755/1503] 10.222.0.15 () {54 vars in 964 bytes} [Thu Sep 15 00:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:40:59.497067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eb86f-9934-439c-9c56-aa23a86077a0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.497348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eb86f-9934-439c-9c56-aa23a86077a0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.497618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eb86f-9934-439c-9c56-aa23a86077a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.528548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:59.573194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-127eb86f-9934-439c-9c56-aa23a86077a0 None None] MFA Rules not processed for user `7727b52cd17b41708b243dac7b4f2c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:40:59.574380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-127eb86f-9934-439c-9c56-aa23a86077a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:59.650596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 749/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Thu Sep 15 00:40:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:40:59.658229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1d18c-5984-4cb8-88d4-916021790ea7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:40:59.659733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-04a1d18c-5984-4cb8-88d4-916021790ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:59.709136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:59.763117 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1d18c-5984-4cb8-88d4-916021790ea7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:40:59.836915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:40:59.837515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:40:59.837790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:40:59.838058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:40:59.839688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:40:59.939130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:40:59.940502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:40:59.944491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:40:59.959967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 756/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:40:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:41:00.004322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a850b-1a73-4ff6-8676-8e7db62a5e95 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:41:00.004607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a850b-1a73-4ff6-8676-8e7db62a5e95 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:41:00.004876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a850b-1a73-4ff6-8676-8e7db62a5e95 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:41:00.007039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 750/1506] 10.222.0.15 () {52 vars in 830 bytes} [Thu Sep 15 00:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:41:00.010557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:41:00.011809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:41:00.107778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f08368-df34-4f52-830c-fe99bfb4ca46 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:41:00.201059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:41:00.201599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:41:00.201872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:41:00.202138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] PATH_INFO: `/v3/users/7b0c7516e6374410a65616ad3f9193f9` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:41:00.214052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0c7516e6374410a65616ad3f9193f9)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:41:00.215048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:41:00.215048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:41:00.215048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:41:00.216278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:41:00.226204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:41:00.282683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0c7516e6374410a65616ad3f9193f9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:00.292150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0c7516e6374410a65616ad3f9193f9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:00.292744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0c7516e6374410a65616ad3f9193f9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:00.309804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0c7516e6374410a65616ad3f9193f9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:00.310250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0c7516e6374410a65616ad3f9193f9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:00.339003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 757/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:41:00 2022] DELETE /identity/v3/users/7b0c7516e6374410a65616ad3f9193f9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:41:00.343344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:41:00.343831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:41:00.343947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:41:00.400370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:41:00.401569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:41:00.475664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:41:00.487902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 751/1508] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:41:00.498566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:41:00.499843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:41:00.596776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d1ac6-20c2-4884-8904-381dd98722c8 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:41:00.651297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:41:00.671594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:41:00.672206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:41:00.672486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:41:00.672762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:41:00.674882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:41:00.783868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:41:00.785213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e646e-a907-4202-88bf-00a23685a592 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:41:00.802631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 758/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:41:01.164612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:41:01.165932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:41:01.178851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be533faa-c0f3-4bed-8bd0-96bec47bb886 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:41:01.278501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:41:01.279148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:41:01.279419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:41:01.279717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] PATH_INFO: `/v3/projects/3baf00ec8bbb40ee87f6ce698f331fed` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:41:01.289851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3baf00ec8bbb40ee87f6ce698f331fed)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:41:01.290863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:41:01.290863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:41:01.290863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:41:01.292093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:41:01.350912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3baf00ec8bbb40ee87f6ce698f331fed', 'request_id': 'req-23d6b714-fd54-44bb-9d64-5eed13f25e2b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:01.356910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3baf00ec8bbb40ee87f6ce698f331fed', 'request_id': 'req-23d6b714-fd54-44bb-9d64-5eed13f25e2b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:01.365760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3baf00ec8bbb40ee87f6ce698f331fed', 'request_id': 'req-23d6b714-fd54-44bb-9d64-5eed13f25e2b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:01.396215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] The token cache is being invalidate because project 3baf00ec8bbb40ee87f6ce698f331fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:41:01.396504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:01.396938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23d6b714-fd54-44bb-9d64-5eed13f25e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:41:01.398320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 752/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:41:01 2022] DELETE /identity/v3/projects/3baf00ec8bbb40ee87f6ce698f331fed => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:41:10.800383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=62 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:41:11.394092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:42:30.585304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ccf3a-9e87-4871-bc2b-3d12af9ef8f8 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:30.585897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ccf3a-9e87-4871-bc2b-3d12af9ef8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:30.585994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ccf3a-9e87-4871-bc2b-3d12af9ef8f8 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:30.587305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:30.592175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44b3b0-fe1e-4f2f-8c6d-9d20e8356baa None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:30.592574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44b3b0-fe1e-4f2f-8c6d-9d20e8356baa None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:30.592716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44b3b0-fe1e-4f2f-8c6d-9d20e8356baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:30.672583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-bb44b3b0-fe1e-4f2f-8c6d-9d20e8356baa None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:30.673724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb44b3b0-fe1e-4f2f-8c6d-9d20e8356baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:30.780291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:42:30.933591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ccf3a-9e87-4871-bc2b-3d12af9ef8f8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:30.935720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a35ccf3a-9e87-4871-bc2b-3d12af9ef8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:31.051929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ccf3a-9e87-4871-bc2b-3d12af9ef8f8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:31.057362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2275c05-9cb2-4d81-b4fd-bb15cdfac8b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:31.057945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275c05-9cb2-4d81-b4fd-bb15cdfac8b8 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:31.058222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275c05-9cb2-4d81-b4fd-bb15cdfac8b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:31.058496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275c05-9cb2-4d81-b4fd-bb15cdfac8b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:31.066423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2275c05-9cb2-4d81-b4fd-bb15cdfac8b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:31.068510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2275c05-9cb2-4d81-b4fd-bb15cdfac8b8 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:31.077691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 760/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:32.043574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11aee8-d2d1-4cd5-b3d9-10e12c536e35 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:32.044177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11aee8-d2d1-4cd5-b3d9-10e12c536e35 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:32.044177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11aee8-d2d1-4cd5-b3d9-10e12c536e35 None None] PATH_INFO: `/v3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:32.045445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 754/1514] 10.222.0.18 () {54 vars in 925 bytes} [Thu Sep 15 00:42:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:42:32.052025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:32.052504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:32.052611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:32.112913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:32.114105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:32.154732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ec8a0-eecc-421f-8e90-c00797b50ca1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:32.155020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ec8a0-eecc-421f-8e90-c00797b50ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:32.155298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ec8a0-eecc-421f-8e90-c00797b50ca1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:32.156709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 755/1515] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:42:32.163531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820a6a-ccbe-4377-af7e-7c6a635e9da7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:32.163820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820a6a-ccbe-4377-af7e-7c6a635e9da7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:32.164151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820a6a-ccbe-4377-af7e-7c6a635e9da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:32.179792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:42:32.184568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-49820a6a-ccbe-4377-af7e-7c6a635e9da7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:32.185721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-49820a6a-ccbe-4377-af7e-7c6a635e9da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:32.185924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 761/1516] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Sep 15 00:42:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:42:32.193779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 756/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:42:32.346435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:32.358368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ecfe9-496f-4aae-995f-11b5cd57cb95 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:32.363789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbcc0f-b227-4ac3-ad0f-e9f889230499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:32.364382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbcc0f-b227-4ac3-ad0f-e9f889230499 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:32.364658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbcc0f-b227-4ac3-ad0f-e9f889230499 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:32.364929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbcc0f-b227-4ac3-ad0f-e9f889230499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:32.372871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbcc0f-b227-4ac3-ad0f-e9f889230499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:32.374435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbcc0f-b227-4ac3-ad0f-e9f889230499 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:32.384671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 762/1518] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:41.173453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af57110-eb59-4c53-8811-843e1d1f0b3b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:41.173872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af57110-eb59-4c53-8811-843e1d1f0b3b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:41.174025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af57110-eb59-4c53-8811-843e1d1f0b3b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:41.175408 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 757/1519] 10.222.0.15 () {54 vars in 961 bytes} [Thu Sep 15 00:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:41.180494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82b661-3658-4405-adc1-db369248851b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:41.180878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82b661-3658-4405-adc1-db369248851b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:41.181037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82b661-3658-4405-adc1-db369248851b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:41.200121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-0b82b661-3658-4405-adc1-db369248851b None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:41.201365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0b82b661-3658-4405-adc1-db369248851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:41.210291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 763/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Thu Sep 15 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:42:41.217854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af57110-eb59-4c53-8811-843e1d1f0b3b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:41.231651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af57110-eb59-4c53-8811-843e1d1f0b3b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:41.236527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf26f4-415e-4089-a454-5112f2734f7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:41.237108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf26f4-415e-4089-a454-5112f2734f7c service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:41.237402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf26f4-415e-4089-a454-5112f2734f7c service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:41.237653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf26f4-415e-4089-a454-5112f2734f7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:41.245897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf26f4-415e-4089-a454-5112f2734f7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:41.247465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf26f4-415e-4089-a454-5112f2734f7c service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:41.257734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 758/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:43.855120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e39a3-3980-496d-8a1d-2d47a9542ef6 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:43.855690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e39a3-3980-496d-8a1d-2d47a9542ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:43.855690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e39a3-3980-496d-8a1d-2d47a9542ef6 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:43.857026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:43.863277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9176e-2806-465d-b8c9-332184913182 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:43.863741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9176e-2806-465d-b8c9-332184913182 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:43.863843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9176e-2806-465d-b8c9-332184913182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:43.884845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f0a9176e-2806-465d-b8c9-332184913182 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:43.886052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f0a9176e-2806-465d-b8c9-332184913182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:43.893918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:42:44.052506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e39a3-3980-496d-8a1d-2d47a9542ef6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:44.066884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e39a3-3980-496d-8a1d-2d47a9542ef6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:44.072894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66ca99-4c98-4d68-b56e-3160cf68c141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:44.073472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66ca99-4c98-4d68-b56e-3160cf68c141 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:44.073763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66ca99-4c98-4d68-b56e-3160cf68c141 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:44.074011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66ca99-4c98-4d68-b56e-3160cf68c141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:44.081793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66ca99-4c98-4d68-b56e-3160cf68c141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:44.083346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66ca99-4c98-4d68-b56e-3160cf68c141 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:44.092791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 765/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:45.163131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c219-fef3-4530-91d5-7c1fc56954c9 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:45.163696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c219-fef3-4530-91d5-7c1fc56954c9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:45.163696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c219-fef3-4530-91d5-7c1fc56954c9 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:45.165113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 00:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:45.171359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a3b03-45a2-4ea6-82b2-1c83082b838a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:45.171830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a3b03-45a2-4ea6-82b2-1c83082b838a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:45.171964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a3b03-45a2-4ea6-82b2-1c83082b838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:45.190958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5b9a3b03-45a2-4ea6-82b2-1c83082b838a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:45.192181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5b9a3b03-45a2-4ea6-82b2-1c83082b838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:45.200490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 00:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:42:45.352405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c219-fef3-4530-91d5-7c1fc56954c9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:45.367212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c219-fef3-4530-91d5-7c1fc56954c9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:45.372693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd1692-b38d-4e46-b5f4-513fc4fa42c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:45.373285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd1692-b38d-4e46-b5f4-513fc4fa42c7 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:45.373559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd1692-b38d-4e46-b5f4-513fc4fa42c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:45.373827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd1692-b38d-4e46-b5f4-513fc4fa42c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:45.382549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd1692-b38d-4e46-b5f4-513fc4fa42c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:45.384164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd1692-b38d-4e46-b5f4-513fc4fa42c7 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:45.393205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 761/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:49.410331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198992-4754-4b02-853c-12a38b7dbb50 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:49.410869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198992-4754-4b02-853c-12a38b7dbb50 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:49.410869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198992-4754-4b02-853c-12a38b7dbb50 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:49.412290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:49.417432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b10d-92c6-496a-82e7-eeceb82256bb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:49.417822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b10d-92c6-496a-82e7-eeceb82256bb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:49.417963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b10d-92c6-496a-82e7-eeceb82256bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:49.437683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-43b5b10d-92c6-496a-82e7-eeceb82256bb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:49.438888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-43b5b10d-92c6-496a-82e7-eeceb82256bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:49.447111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:42:49.734555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198992-4754-4b02-853c-12a38b7dbb50 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:49.736264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5c198992-4754-4b02-853c-12a38b7dbb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:49.814905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198992-4754-4b02-853c-12a38b7dbb50 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:49.818531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67643914-5f4a-46c3-81df-03eb0cb295f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:49.819171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67643914-5f4a-46c3-81df-03eb0cb295f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:49.819452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67643914-5f4a-46c3-81df-03eb0cb295f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:49.819759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67643914-5f4a-46c3-81df-03eb0cb295f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:49.828958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67643914-5f4a-46c3-81df-03eb0cb295f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:49.830404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67643914-5f4a-46c3-81df-03eb0cb295f3 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:49.840094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 768/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:52.043851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b7dce-f477-498c-b77d-26c4b14b3d2b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:52.044368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b7dce-f477-498c-b77d-26c4b14b3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:52.044368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b7dce-f477-498c-b77d-26c4b14b3d2b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:52.045666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:52.050914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f829f1-5dab-44ee-90f5-4d0ac94e1688 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:52.051350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f829f1-5dab-44ee-90f5-4d0ac94e1688 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:52.051452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f829f1-5dab-44ee-90f5-4d0ac94e1688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:52.071128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a5f829f1-5dab-44ee-90f5-4d0ac94e1688 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:52.072336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a5f829f1-5dab-44ee-90f5-4d0ac94e1688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:52.079957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:42:52.403416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b7dce-f477-498c-b77d-26c4b14b3d2b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:52.408770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:42:52.415833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b7dce-f477-498c-b77d-26c4b14b3d2b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:52.419318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e7b0d-2bdf-45c7-9038-0d3184845866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:52.419944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e7b0d-2bdf-45c7-9038-0d3184845866 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:52.420215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e7b0d-2bdf-45c7-9038-0d3184845866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:52.420479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e7b0d-2bdf-45c7-9038-0d3184845866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:52.429247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e7b0d-2bdf-45c7-9038-0d3184845866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:52.430687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e7b0d-2bdf-45c7-9038-0d3184845866 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:52.440197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 764/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:42:56.145128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00349cc-3747-490f-88b9-752b2f337798 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:56.145674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00349cc-3747-490f-88b9-752b2f337798 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:56.145674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00349cc-3747-490f-88b9-752b2f337798 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:56.147049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:56.152167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737bae3-2b96-44fa-9f0b-6ae91dc9a3eb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:56.152580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737bae3-2b96-44fa-9f0b-6ae91dc9a3eb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:56.152701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737bae3-2b96-44fa-9f0b-6ae91dc9a3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:56.171633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3737bae3-2b96-44fa-9f0b-6ae91dc9a3eb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:56.172808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3737bae3-2b96-44fa-9f0b-6ae91dc9a3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:56.180868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:42:56.479422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00349cc-3747-490f-88b9-752b2f337798 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:56.481131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a00349cc-3747-490f-88b9-752b2f337798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:56.558923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00349cc-3747-490f-88b9-752b2f337798 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:56.562504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925387e5-d592-487f-b761-ce175e2a0324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:56.563095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925387e5-d592-487f-b761-ce175e2a0324 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:56.563371 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925387e5-d592-487f-b761-ce175e2a0324 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:56.563665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925387e5-d592-487f-b761-ce175e2a0324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:56.572273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925387e5-d592-487f-b761-ce175e2a0324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:56.573680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925387e5-d592-487f-b761-ce175e2a0324 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:56.582785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 771/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:42:58.689618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008360d-88a8-4a83-b929-c4bfb7ba7c1b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:58.690068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008360d-88a8-4a83-b929-c4bfb7ba7c1b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:58.690198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008360d-88a8-4a83-b929-c4bfb7ba7c1b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:58.691596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 00:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:42:58.696859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece2d0-23ac-407b-82f9-063396304684 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:58.697262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece2d0-23ac-407b-82f9-063396304684 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:58.697422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece2d0-23ac-407b-82f9-063396304684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:58.716085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7fece2d0-23ac-407b-82f9-063396304684 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:42:58.717253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7fece2d0-23ac-407b-82f9-063396304684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:58.725411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 00:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:42:59.023227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008360d-88a8-4a83-b929-c4bfb7ba7c1b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:42:59.024936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d008360d-88a8-4a83-b929-c4bfb7ba7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:42:59.038256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008360d-88a8-4a83-b929-c4bfb7ba7c1b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:42:59.041691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62979a0d-2454-41f6-a699-6a10181a7c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:42:59.042274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979a0d-2454-41f6-a699-6a10181a7c13 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:42:59.042552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979a0d-2454-41f6-a699-6a10181a7c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:42:59.042819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979a0d-2454-41f6-a699-6a10181a7c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:42:59.051508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62979a0d-2454-41f6-a699-6a10181a7c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:42:59.052958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62979a0d-2454-41f6-a699-6a10181a7c13 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:42:59.062361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 767/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:43:08.711039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:43:09.058179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=14 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:25.140355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4b28-d171-45d4-af10-0362195d1909 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.141024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4b28-d171-45d4-af10-0362195d1909 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.141024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4b28-d171-45d4-af10-0362195d1909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.164564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4e6e4b28-d171-45d4-af10-0362195d1909 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.165848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4e6e4b28-d171-45d4-af10-0362195d1909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.176608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 773/1540] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.188048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69962a-a555-4f47-8820-f534b92fd3e1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.188675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69962a-a555-4f47-8820-f534b92fd3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.188675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69962a-a555-4f47-8820-f534b92fd3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.203284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a23df-ff81-4688-8348-da6baf25984b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.203624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a23df-ff81-4688-8348-da6baf25984b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.203860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a23df-ff81-4688-8348-da6baf25984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.225730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0e69962a-a555-4f47-8820-f534b92fd3e1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.227927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0e69962a-a555-4f47-8820-f534b92fd3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.248970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-153a23df-ff81-4688-8348-da6baf25984b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.253303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-153a23df-ff81-4688-8348-da6baf25984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.256526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 768/1541] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.257861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.258148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.258423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.276027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 774/1542] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.282204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35ab9-7d6f-4227-860f-b919301a7de1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.282786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35ab9-7d6f-4227-860f-b919301a7de1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.283062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35ab9-7d6f-4227-860f-b919301a7de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.300268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.301487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.310155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 769/1543] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.310326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7dd35ab9-7d6f-4227-860f-b919301a7de1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.310970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.311704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7dd35ab9-7d6f-4227-860f-b919301a7de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.321324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8b9e9-95aa-4c13-8e12-58d9cae7ff77 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.326570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.327204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.327518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.327793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.328446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.328749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.330390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9204d-2789-4791-a601-8afe139f0776 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.330807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 775/1544] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.332453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea0d0-2cfd-4287-8cd3-6f1e4daf1fba None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.332744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea0d0-2cfd-4287-8cd3-6f1e4daf1fba None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.333019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea0d0-2cfd-4287-8cd3-6f1e4daf1fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.344648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 770/1545] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.346519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f99c8d-e41a-4b8f-a42c-5b86de53068a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.347729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f99c8d-e41a-4b8f-a42c-5b86de53068a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.348405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f99c8d-e41a-4b8f-a42c-5b86de53068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.353609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-589ea0d0-2cfd-4287-8cd3-6f1e4daf1fba None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.354649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-589ea0d0-2cfd-4287-8cd3-6f1e4daf1fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.363324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 776/1546] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.364574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b02fc-ada6-4edf-a0df-80169ce1c497 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.364857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b02fc-ada6-4edf-a0df-80169ce1c497 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.365128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b02fc-ada6-4edf-a0df-80169ce1c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.372567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d6f99c8d-e41a-4b8f-a42c-5b86de53068a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.373767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d6f99c8d-e41a-4b8f-a42c-5b86de53068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.381959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 771/1547] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.383140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ccec-3aa8-4611-b802-b008df84be9c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.383418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ccec-3aa8-4611-b802-b008df84be9c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.383732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ccec-3aa8-4611-b802-b008df84be9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.388585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-da2b02fc-ada6-4edf-a0df-80169ce1c497 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.389789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-da2b02fc-ada6-4edf-a0df-80169ce1c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.398444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 777/1548] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.399713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.400091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.400286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.403910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4819ccec-3aa8-4611-b802-b008df84be9c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.405105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4819ccec-3aa8-4611-b802-b008df84be9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.419748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 772/1549] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.419748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91798823-7929-4def-8ea1-fafc1016abe5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.419748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91798823-7929-4def-8ea1-fafc1016abe5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.419748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91798823-7929-4def-8ea1-fafc1016abe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.423187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.424421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.432319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 778/1550] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.433128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.439556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-91798823-7929-4def-8ea1-fafc1016abe5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.440874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-91798823-7929-4def-8ea1-fafc1016abe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.447622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5b90c-b1cd-400e-9fb0-70fe37f175af None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.448792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 773/1551] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.452477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a7814-95ec-4f6d-82f5-2c7975869be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.452918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.453063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a7814-95ec-4f6d-82f5-2c7975869be8 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.453198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.453355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a7814-95ec-4f6d-82f5-2c7975869be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.453493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.453596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a7814-95ec-4f6d-82f5-2c7975869be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.454213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a7814-95ec-4f6d-82f5-2c7975869be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.456710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a7814-95ec-4f6d-82f5-2c7975869be8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.473724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.474900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.482663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 774/1552] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.483431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.492874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fda0d9-35e6-4754-9a66-6560f1e9d397 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.495553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 779/1553] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.496683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52c4a-88f1-4aa5-959c-ffb131993741 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.496970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52c4a-88f1-4aa5-959c-ffb131993741 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.497247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52c4a-88f1-4aa5-959c-ffb131993741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.500420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.501033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.501309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.501574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.502214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.502511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.504652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7c1d-8869-4d08-966a-63c3a81b78e1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.516785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 775/1554] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.517425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bbc52c4a-88f1-4aa5-959c-ffb131993741 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.518365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.519409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.519650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bbc52c4a-88f1-4aa5-959c-ffb131993741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.519748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.539540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.540744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.579468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 780/1555] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.584394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.584683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.584963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.600704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 776/1556] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.601683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.610071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.611271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.612865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e1f33-0bf5-4c94-862b-ee6bb329a5a8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.617551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.618146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.618426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.618692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.619334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.619671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.619856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 781/1557] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.620654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.621255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.630617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dc9a0-85ad-40c4-9350-7500691c21e4 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.634383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 777/1558] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.641356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.641456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.641456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.641456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.641456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.641826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.641826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.641826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:44:25.641826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:44:25.641826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:44:25.642245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.652883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb568ad-a611-418e-b42e-fa70f7238b14 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.655216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 782/1559] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.656157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.658641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.659509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.659786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.660061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.660708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.661014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.662803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.666778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f8705-878e-4d1a-be73-2a9b4f17b4a0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.671377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.671996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.672277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.672538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.673161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.674626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:44:25.674626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:44:25.674626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:44:25.675445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.679029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 778/1560] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.679977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.689586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c5f1-343b-4f20-a391-c3c3cdbfbf32 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.695963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.696833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.697119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.697387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.698076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.699094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:44:25.699094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:44:25.699094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:44:25.700360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159febd-84cc-45d9-b870-8711220c8479 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.704012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 783/1561] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/users => generated 296 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.704855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.717598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29372e3-06f8-4710-9bf1-5672cd5728da admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.723087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.723727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.724007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.724272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.724922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.725223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.726804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.735585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 779/1562] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.735585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.735585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.735585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.746132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 784/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.746132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.751586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec101b-d29b-4600-adc9-a2b05ea4b7ad admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.759618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:25.760830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:25.789613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:25.803653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:25.810037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.810640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.810925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.811194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.811876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:25.812178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.813773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.822899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 780/1564] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:25.823723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.833096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 785/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:25.833096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.833884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba7c5-0637-4f07-8b4a-70fcc956d563 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.838363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18129c-7601-44d5-b4c2-4d0f9424328a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.838498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.839208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.839503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.839816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.840444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.842351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.843635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.844219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.844496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.844764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.845381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.847283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.871432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 781/1566] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.872482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.874822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 786/1567] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.875725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.934680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c0741-1acc-4430-b070-715913be929c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.936597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8a67-3b87-4e71-a0d2-38515d577a97 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.939234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.939866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.940146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.940956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.941587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.941690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.942392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.942673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.943034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.943535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.943753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.945320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.955985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 787/1568] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:25.956681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.966824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543947f-7c8f-4fb9-bd1d-2bc02ec4e7f2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:25.971425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:25.974952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 782/1569] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.975089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:25.975089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:25.975089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:25.975372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:25.975459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:25.975459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:25.998301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 788/1570] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:25.999126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.010783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ef10a-fd7c-46f9-8a16-ca73aecd2b39 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.042215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764c67-e983-4ebe-ac2d-c2281ea1d730 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.047241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.047865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.048132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.048541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.049022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.051646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.070303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.071142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.071417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.071724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.072404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:26.072829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.074524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.079232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 783/1571] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.079935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.088618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 789/1572] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:26.088618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.101879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b12b33-04ad-4574-8a08-0906f1a05afc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.118985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:26.121561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:26.142884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a1576-ca2c-449d-8cfc-89c3f5b32a71 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.147613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.148209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.148484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.148758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.149379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.151619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.161049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.161663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.162008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.162677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.163340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.165285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.177578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 784/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.178323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.190193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 790/1574] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.190193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.198847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f2e44-f565-4c1e-8e37-222aa3ccd023 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.246396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62013dd4-028f-4aff-ad07-2bd282400dd7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.251135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.251765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.252048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.252317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.253515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.255443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.255766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.256659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.257045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.257336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.258144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.259759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.270325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 791/1575] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.271218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.283173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 785/1576] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.283333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb784555-0132-4267-95ce-380b05a01fb3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.283444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.285926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.286516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.286784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.287057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.287713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.289598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.290556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e56b-ad24-4fcf-8263-2de7b3261566 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.295012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.295609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.295885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.296175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.296811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.298406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.313027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 786/1577] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.313966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 792/1578] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/users => generated 279 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.315108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.315517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.326908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9781ee-23b2-4eb8-8ac3-ffb1f93fee7d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.327098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10945c17-9e6a-468e-b15d-b94095746ff5 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.331625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.332231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.332510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.332798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.333223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.333413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.333826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.334108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.334387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.335050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.335330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.336961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.358615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 787/1579] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.359527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.369575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d5f92-08d2-41f3-91a9-3c176f27db2a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.370684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 793/1580] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.371360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.427249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.427876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.428152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.428429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.429048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.430962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.432193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018d880-a626-4de8-907b-f9e5ed7bbeff admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.436811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.437406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.437685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.437957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.438611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.440283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.455451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 794/1581] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.455451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.460871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 788/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.460871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.463942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1f849-b25e-4c67-b89f-cbbed3122d13 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.469280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.469479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33262e-b6f1-4945-a5c6-1b3a6af8b1dd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.469926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.470207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.470477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] PATH_INFO: `/v3/projects/7090f91f78534107ba4b5ccd7f80e145/users/4e0e6a33679f4cac96704f71dcfa5c01/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.473213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:26.474717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.475523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.475788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.476059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.476686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.478740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.483908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:26.488822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7090f91f78534107ba4b5ccd7f80e145, user_id=4e0e6a33679f4cac96704f71dcfa5c01, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.491139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 789/1583] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.491715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.492298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.503400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d6547-7edb-4457-b98d-f6c3af8c49bf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.505052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 795/1584] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:26 2022] PUT /identity/v3/projects/7090f91f78534107ba4b5ccd7f80e145/users/4e0e6a33679f4cac96704f71dcfa5c01/roles/87f1831257824560813f954348534540 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:26.508827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.559696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.560321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.560599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.560872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.561685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.563290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.572196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98da13d-e139-46a9-a311-2ca81efb2493 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.573141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 790/1585] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.573969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.576976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.577696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.578059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.578397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.579076 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.580728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.585126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa83701-4e21-45e1-af7c-ff06b3fea346 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.589956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.590537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.590817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.591092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.591185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 796/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.591905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.592044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.593647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.602363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be8c7-09c1-4982-b8ba-8a30b4a0fcff admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.607230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 791/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.607230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.607473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.607988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.608290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.608558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.609203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.611287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.615416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa075d-3fdb-4793-9bfb-be508f936d04 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.620356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.621171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.621447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.621730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.622392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.624044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.636576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 797/1588] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:26.640963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 792/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.641119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.641242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.650564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12aaab-5770-4718-8e49-ebb0315d5826 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.653093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ada79-fc47-468f-9f6e-c31551f4240a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.655571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cbcae-01c6-4eff-8a98-82e0c136b76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.656182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cbcae-01c6-4eff-8a98-82e0c136b76e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.656472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cbcae-01c6-4eff-8a98-82e0c136b76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.656730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cbcae-01c6-4eff-8a98-82e0c136b76e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.657372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cbcae-01c6-4eff-8a98-82e0c136b76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.657717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.658316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.658594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.658867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.658981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cbcae-01c6-4eff-8a98-82e0c136b76e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.660147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.661751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.671922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 798/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.672072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 793/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.675646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.675801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.675801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.675801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.684575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b415a7-f80e-4dae-8c6c-67b7d0668cf9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.689198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.689837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.690118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.690482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] PATH_INFO: `/v3/projects/67568d58a68e4837a58c5432a01ca9fa/users/6401018aba8442ceac151bc6849fb4b2/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.691935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:26.693121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:26.700663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 799/1592] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:26.701431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.709309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67568d58a68e4837a58c5432a01ca9fa, user_id=6401018aba8442ceac151bc6849fb4b2, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.710358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:44:26.710358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:44:26.710358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:44:26.711730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86f7c-9750-424a-b9ed-b2b9f4e9c3ac None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.712445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.716310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.716983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.717268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.717594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.718292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.720120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.726806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 794/1593] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:26 2022] PUT /identity/v3/projects/67568d58a68e4837a58c5432a01ca9fa/users/6401018aba8442ceac151bc6849fb4b2/roles/87f1831257824560813f954348534540 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:26.727713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.730873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 800/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.731868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.743062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4858d-5b76-41e0-8ea9-1390945d506f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.797629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15f13d-e4ad-43ee-a722-8b57a7847d4b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.801057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.801658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.801930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.802194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.802307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.802806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.802902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.803082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.803353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.804029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.804438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.805608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.814356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 801/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.815267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.816707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 795/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.817754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.825384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179a02-4ebf-470e-a2b8-d09d14857e5d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.827315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99565f-fd99-4578-b042-ca5509fc5b09 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.829905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.830531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.830804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.831638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] PATH_INFO: `/v3/projects/d0742191cd4d45d0a67eaf82b377e2c3/users/38af5c5ada9644f59dd85ed674828f37/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.832397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.832959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.833240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.833512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.834317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.835926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.845715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 796/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:26.846651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.847422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0742191cd4d45d0a67eaf82b377e2c3, user_id=38af5c5ada9644f59dd85ed674828f37, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.850314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.858777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e33e0-7071-4f48-8492-cf318136312a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.866056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 802/1598] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:26 2022] PUT /identity/v3/projects/d0742191cd4d45d0a67eaf82b377e2c3/users/38af5c5ada9644f59dd85ed674828f37/roles/87f1831257824560813f954348534540 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:26.867137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.869074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:26.876231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:26.883923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:26.921690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.922670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.922946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.923212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] PATH_INFO: `/v3/projects/bc126ea1c77342d091a96baf33bedf6e/users/33070b58c99741eeb9f97a73507d012e/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.930089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b941f8d-8df1-4f24-9235-fd62101059e8 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:26.933861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef4a42-5325-47ed-8dfe-b25afdd7ed60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:26.934458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef4a42-5325-47ed-8dfe-b25afdd7ed60 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.934732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef4a42-5325-47ed-8dfe-b25afdd7ed60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.935158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef4a42-5325-47ed-8dfe-b25afdd7ed60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.938352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc126ea1c77342d091a96baf33bedf6e, user_id=33070b58c99741eeb9f97a73507d012e, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.941159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.944909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef4a42-5325-47ed-8dfe-b25afdd7ed60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:26.946298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef4a42-5325-47ed-8dfe-b25afdd7ed60 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:26.954998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 797/1599] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:26 2022] PUT /identity/v3/projects/bc126ea1c77342d091a96baf33bedf6e/users/33070b58c99741eeb9f97a73507d012e/roles/87f1831257824560813f954348534540 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:26.956405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:26.957070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 803/1600] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:26.958337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:26.958558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:26.958865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:26.989171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] MFA Rules not processed for user `6401018aba8442ceac151bc6849fb4b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:26.990623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.028103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280deb-acdb-4785-a136-7f8e3919b078 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.033753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.034465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.034826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.035036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] PATH_INFO: `/v3/projects/fe33d8cbe3e04bdeb9c9691e048ba925/users/eff3aa87f49348e2ad73c520810bc88f/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.053867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe33d8cbe3e04bdeb9c9691e048ba925, user_id=eff3aa87f49348e2ad73c520810bc88f, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.056964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.068167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 804/1601] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.069439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.082791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1ce8c-ff0a-4a6f-95f8-315e3030d775 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.084105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 798/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:26 2022] PUT /identity/v3/projects/fe33d8cbe3e04bdeb9c9691e048ba925/users/eff3aa87f49348e2ad73c520810bc88f/roles/87f1831257824560813f954348534540 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:27.095984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.164742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3605b-76fd-45c4-9ec1-39c037274c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.164742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3605b-76fd-45c4-9ec1-39c037274c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.164742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3605b-76fd-45c4-9ec1-39c037274c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.164742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3605b-76fd-45c4-9ec1-39c037274c21 admin admin] PATH_INFO: `/v3/projects/6a9d9cf3606845ec964dd8320126293e/users/c5307118357e44ffbff5074ae4a94c9d/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.179034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb1f8-328f-4b83-89ad-67aabd6e43a2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.182067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3605b-76fd-45c4-9ec1-39c037274c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9d9cf3606845ec964dd8320126293e, user_id=c5307118357e44ffbff5074ae4a94c9d, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.184033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.184639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.184916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.185240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.185432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3605b-76fd-45c4-9ec1-39c037274c21 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.185897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.187542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.198913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 799/1603] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:27.200384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.208302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 805/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:27 2022] PUT /identity/v3/projects/6a9d9cf3606845ec964dd8320126293e/users/c5307118357e44ffbff5074ae4a94c9d/roles/87f1831257824560813f954348534540 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:27.210448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc4f39-1470-4ab8-8798-019b47f360f2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.210805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc4f39-1470-4ab8-8798-019b47f360f2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.211156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc4f39-1470-4ab8-8798-019b47f360f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.213714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07923794-2aad-4d3f-90d6-82309f5078c5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.238070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:27.246240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-69fc4f39-1470-4ab8-8798-019b47f360f2 None None] MFA Rules not processed for user `38af5c5ada9644f59dd85ed674828f37`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:27.248309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-69fc4f39-1470-4ab8-8798-019b47f360f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.265406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:27.282518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1bd71-37b1-4138-85a9-45d77f0693de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.284109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1bd71-37b1-4138-85a9-45d77f0693de admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.284550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1bd71-37b1-4138-85a9-45d77f0693de admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.285000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1bd71-37b1-4138-85a9-45d77f0693de admin admin] PATH_INFO: `/v3/projects/b77ee72d79e048be9cb68bf296ec6bec/users/5cf5ef2e256f426ca72574e40caf1ecd/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.305077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1bd71-37b1-4138-85a9-45d77f0693de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77ee72d79e048be9cb68bf296ec6bec, user_id=5cf5ef2e256f426ca72574e40caf1ecd, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.308175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1bd71-37b1-4138-85a9-45d77f0693de admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.323394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 800/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:27 2022] PUT /identity/v3/projects/b77ee72d79e048be9cb68bf296ec6bec/users/5cf5ef2e256f426ca72574e40caf1ecd/roles/87f1831257824560813f954348534540 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:27.324199 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4311-610d-4cd6-a35c-350766e64c7e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.324492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4311-610d-4cd6-a35c-350766e64c7e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.324756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4311-610d-4cd6-a35c-350766e64c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.337248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 806/1606] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.338649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17eba09-92e3-4605-ab27-d0c4b3926b9f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.339134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17eba09-92e3-4605-ab27-d0c4b3926b9f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.339414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17eba09-92e3-4605-ab27-d0c4b3926b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.374428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1a3e4311-610d-4cd6-a35c-350766e64c7e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:27.375665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1a3e4311-610d-4cd6-a35c-350766e64c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.378865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e17eba09-92e3-4605-ab27-d0c4b3926b9f None None] MFA Rules not processed for user `eff3aa87f49348e2ad73c520810bc88f`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:27.380354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e17eba09-92e3-4605-ab27-d0c4b3926b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.462376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 807/1607] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.464689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47881fc-0e54-4d45-b0e0-e0565b5aea61 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.464985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47881fc-0e54-4d45-b0e0-e0565b5aea61 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.465258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47881fc-0e54-4d45-b0e0-e0565b5aea61 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.467314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 808/1608] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 00:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:27.468061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47881fc-0e54-4d45-b0e0-e0565b5aea61 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.489208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 801/1609] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.490650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c029c-5033-4d9e-b150-3d34e6a34b5e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.491150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c029c-5033-4d9e-b150-3d34e6a34b5e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.492125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c029c-5033-4d9e-b150-3d34e6a34b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.517078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-de8c029c-5033-4d9e-b150-3d34e6a34b5e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:27.518909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-de8c029c-5033-4d9e-b150-3d34e6a34b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.530698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 802/1610] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.530698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81ba7-0464-4d26-b70b-30a8f28a4fa2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.530698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81ba7-0464-4d26-b70b-30a8f28a4fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.530698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81ba7-0464-4d26-b70b-30a8f28a4fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.566979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c5c81ba7-0464-4d26-b70b-30a8f28a4fa2 None None] MFA Rules not processed for user `5cf5ef2e256f426ca72574e40caf1ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:27.568324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c5c81ba7-0464-4d26-b70b-30a8f28a4fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.571229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47881fc-0e54-4d45-b0e0-e0565b5aea61 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.576950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3952f6-4466-49fa-ae3f-124fb973d435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.579002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3952f6-4466-49fa-ae3f-124fb973d435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.579287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3952f6-4466-49fa-ae3f-124fb973d435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.579729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3952f6-4466-49fa-ae3f-124fb973d435 admin admin] PATH_INFO: `/v3/projects/09628995c5034e75b393d75f0a537404/users/ebda414ffd59490d86758dca0342d353/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.601267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3952f6-4466-49fa-ae3f-124fb973d435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09628995c5034e75b393d75f0a537404, user_id=ebda414ffd59490d86758dca0342d353, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.604190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3952f6-4466-49fa-ae3f-124fb973d435 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.624561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 809/1611] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:27 2022] PUT /identity/v3/projects/09628995c5034e75b393d75f0a537404/users/ebda414ffd59490d86758dca0342d353/roles/87f1831257824560813f954348534540 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:27.626087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33c460-2b2e-4f27-a7b5-132b9a944ab2 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.626372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33c460-2b2e-4f27-a7b5-132b9a944ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.626645 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33c460-2b2e-4f27-a7b5-132b9a944ab2 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.628729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 810/1612] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 00:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:27.629509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33c460-2b2e-4f27-a7b5-132b9a944ab2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.663804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 803/1613] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.665923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.666891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:27.667722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.668000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.716976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:27.717569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:27.721029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33c460-2b2e-4f27-a7b5-132b9a944ab2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.726693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.727314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.727798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.728082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.729386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.731470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.734068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:27.767263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 811/1614] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/projects => generated 357 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:27.768250 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.825652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 804/1615] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:27.826948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:27.850881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e797-cac5-46e9-8f22-1373432eb693 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.856338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9bff7-e631-45fb-b729-a5aa8b0b7af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.857358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9bff7-e631-45fb-b729-a5aa8b0b7af7 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.857652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9bff7-e631-45fb-b729-a5aa8b0b7af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.857952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9bff7-e631-45fb-b729-a5aa8b0b7af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.906657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd83bec-bff8-4dcc-87e1-3435806d5321 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:27.912983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf50143-9a9b-45d3-82dc-920d33f5ccb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:27.913780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf50143-9a9b-45d3-82dc-920d33f5ccb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.914207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf50143-9a9b-45d3-82dc-920d33f5ccb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.914606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf50143-9a9b-45d3-82dc-920d33f5ccb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:27.956037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9bff7-e631-45fb-b729-a5aa8b0b7af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:27.958968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9bff7-e631-45fb-b729-a5aa8b0b7af7 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:27.975722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 812/1616] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:27.978025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:27.979252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:27.979667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.000463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf50143-9a9b-45d3-82dc-920d33f5ccb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.001932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf50143-9a9b-45d3-82dc-920d33f5ccb3 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.015240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] MFA Rules not processed for user `ebda414ffd59490d86758dca0342d353`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:28.015668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 805/1617] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:28.016974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.017253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.017364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:28.017526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.052654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:28.054163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:28.078503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:28.119189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 813/1618] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:28.119189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.135555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:28.141400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43e608-b50b-48a2-a0f6-155f1ab7b3aa None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.148696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.149740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.150125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.150624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.151609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.154136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.164803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 806/1619] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 00:44:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:28.166506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.181331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a872695-eee6-438c-b3a3-a6ad20822f2a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.187376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.188116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.188403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.188741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.191025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 814/1620] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:28 2022] POST /identity/v3/users => generated 285 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:28.192741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.264373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.265904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.276959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 807/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:28.277828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.283954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8d4ec-504c-42d5-9d05-bb875a20bc03 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.291686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.291829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db93020-7c55-4566-9c37-42c31dcffda1 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.292360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.292811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.293395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.298076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.298791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.299183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.299461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.307728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.310204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.346604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 815/1622] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:28.349068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.370265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.372048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.386408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108d52-0f13-4631-a090-c7494e6c018b service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.397707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 808/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:28.401637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.403066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.404815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.406528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.407349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.412334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.430419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 816/1624] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:28.435265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.459535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382800e-07e6-4ab9-93f8-9dff127614f5 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.476277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.477331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.477725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.478109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] PATH_INFO: `/v3/projects/031df15ccfd44744a2bc495b971b1330/users/c1783b7aa9994de5b7f94a7702b0f355/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.508639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031df15ccfd44744a2bc495b971b1330, user_id=c1783b7aa9994de5b7f94a7702b0f355, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.513660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.539849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 809/1625] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:28 2022] PUT /identity/v3/projects/031df15ccfd44744a2bc495b971b1330/users/c1783b7aa9994de5b7f94a7702b0f355/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:44:28.543698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.559791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face2f5b-02cb-41ae-81e3-ee554316820b admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.697455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.698742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.699024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.699298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.700534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.702565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.728955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 817/1626] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:28.729566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.750051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35a1df-b050-439c-80d4-c51ff5ed1344 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.752852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:28.764626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3134d-e40c-49a5-b20a-963e5aa41577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.766116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3134d-e40c-49a5-b20a-963e5aa41577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.766510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3134d-e40c-49a5-b20a-963e5aa41577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.766886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3134d-e40c-49a5-b20a-963e5aa41577 admin admin] PATH_INFO: `/v3/projects/031df15ccfd44744a2bc495b971b1330/users/c1783b7aa9994de5b7f94a7702b0f355/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.774777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3134d-e40c-49a5-b20a-963e5aa41577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031df15ccfd44744a2bc495b971b1330, user_id=c1783b7aa9994de5b7f94a7702b0f355, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:28.778290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3134d-e40c-49a5-b20a-963e5aa41577 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:28.805981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 810/1627] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:28 2022] PUT /identity/v3/projects/031df15ccfd44744a2bc495b971b1330/users/c1783b7aa9994de5b7f94a7702b0f355/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:28.810588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f838f1-e5cf-4308-9d75-88502db2acc8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.811076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f838f1-e5cf-4308-9d75-88502db2acc8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.811660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f838f1-e5cf-4308-9d75-88502db2acc8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.817272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 811/1628] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 00:44:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:28.819990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b12b2d-7850-4daa-9d2d-cea8ce444ad7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.820809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b12b2d-7850-4daa-9d2d-cea8ce444ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.822591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b12b2d-7850-4daa-9d2d-cea8ce444ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:28.852234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:28.874324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4c65f-78d6-42a2-8387-cdefb35a9c73 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:28.902322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-65b12b2d-7850-4daa-9d2d-cea8ce444ad7 None None] MFA Rules not processed for user `c1783b7aa9994de5b7f94a7702b0f355`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:28.904644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-65b12b2d-7850-4daa-9d2d-cea8ce444ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:28.983995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:28.989021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:28.991253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:28.991923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:28.992346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.050640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 812/1629] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:29.052724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:29.053350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:29.053743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.085622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:29.087284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:29.102912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 813/1630] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:44:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:29.104012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:29.118376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:29.120566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:29.140087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 818/1631] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:29.208223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:29.209691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ab21f-c0d1-402f-8380-54c99a7f0a59 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:29.210735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e065fb4-77e5-43e4-982f-7166eb798cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:29.211413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e065fb4-77e5-43e4-982f-7166eb798cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:29.211734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e065fb4-77e5-43e4-982f-7166eb798cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:29.212005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e065fb4-77e5-43e4-982f-7166eb798cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.227694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e065fb4-77e5-43e4-982f-7166eb798cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:29.229512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e065fb4-77e5-43e4-982f-7166eb798cc2 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:29.255443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 814/1632] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:29.256667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032890e-00db-45cf-9901-15403aa8b8ba None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:29.256978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032890e-00db-45cf-9901-15403aa8b8ba None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:29.257222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032890e-00db-45cf-9901-15403aa8b8ba None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.262465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 815/1633] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 00:44:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:29.271904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:29.272435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:29.274297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.299127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:29.300507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:29.312970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 816/1634] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:44:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:29.317183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac3e44-7dbf-440e-b865-7c7d7c763ee2 service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:29.326326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:29.327259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:29.327561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:29.327831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.356524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:29.359766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:29.384306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 819/1635] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:29.461133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:29.474971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df77036-3bc8-48bb-82b3-ba1727489e2a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:29.482738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:29.483413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:29.483734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:29.484006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:29.499807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:29.501596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:29.519979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 817/1636] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:30.053204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:30.065642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7de9f-8fb0-4ca8-b36b-63ed09c442ef service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:30.078953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc517d-c3f8-4406-9c84-4b9946e212f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:30.084029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc517d-c3f8-4406-9c84-4b9946e212f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:30.084310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc517d-c3f8-4406-9c84-4b9946e212f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:30.086131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc517d-c3f8-4406-9c84-4b9946e212f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:30.180410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:30.240067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc517d-c3f8-4406-9c84-4b9946e212f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:30.241503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc517d-c3f8-4406-9c84-4b9946e212f3 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:30.261775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 820/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:30.980936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:31.108776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef557544-12fd-41a8-9c2b-48e605c8b247 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:31.122784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:31.124995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:31.125899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:31.126387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] PATH_INFO: `/v3/users/6401018aba8442ceac151bc6849fb4b2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:31.130480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6401018aba8442ceac151bc6849fb4b2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:31.132080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:44:31.132080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Sep 15 00:44:31.132080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: Sep 15 00:44:31.133817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:31.258830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6401018aba8442ceac151bc6849fb4b2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:31.278126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6401018aba8442ceac151bc6849fb4b2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:31.278846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6401018aba8442ceac151bc6849fb4b2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:31.279631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6401018aba8442ceac151bc6849fb4b2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:31.286052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:31.304712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6401018aba8442ceac151bc6849fb4b2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:31.380442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 818/1638] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:30 2022] DELETE /identity/v3/users/6401018aba8442ceac151bc6849fb4b2 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:31.384186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:31.385068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:31.385471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:31.488678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:31.489974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:31.623516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 821/1639] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:31.649172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:31.651135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:31.844765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44807-0264-4961-8c49-d49512f71f4a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:31.850883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de712f43-5c03-48dd-8666-f7d38a38413b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:31.851458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de712f43-5c03-48dd-8666-f7d38a38413b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:31.852903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de712f43-5c03-48dd-8666-f7d38a38413b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:31.853024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de712f43-5c03-48dd-8666-f7d38a38413b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:31.867302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de712f43-5c03-48dd-8666-f7d38a38413b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:31.868886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de712f43-5c03-48dd-8666-f7d38a38413b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:31.884631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 819/1640] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:33.126061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:33.129152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:33.159584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79f899-685c-48bc-9804-d2475875eed7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:33.166789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:33.167648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:33.168036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:33.168418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] PATH_INFO: `/v3/users/38af5c5ada9644f59dd85ed674828f37` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:33.170556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] RBAC: Authorizing `identity:delete_user(user_id=38af5c5ada9644f59dd85ed674828f37)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:33.172817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:33.301665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38af5c5ada9644f59dd85ed674828f37'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:33.324750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38af5c5ada9644f59dd85ed674828f37'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:33.325586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38af5c5ada9644f59dd85ed674828f37'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:33.338863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38af5c5ada9644f59dd85ed674828f37'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:33.339961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38af5c5ada9644f59dd85ed674828f37'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:33.380105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 822/1641] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:33 2022] DELETE /identity/v3/users/38af5c5ada9644f59dd85ed674828f37 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:33.394985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:33.394985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:33.394985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:33.436111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:33.437365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:33.657078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 820/1642] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:33.936590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:33.938857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:34.075784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:34.092357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca28b1-834e-4bd6-89d5-1a06fb14bfea admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:34.096064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:34.098335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:34.104552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d97cd6-ab31-4ae8-971f-847edf1fcf25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:34.107984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d97cd6-ab31-4ae8-971f-847edf1fcf25 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.109306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d97cd6-ab31-4ae8-971f-847edf1fcf25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.109306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d97cd6-ab31-4ae8-971f-847edf1fcf25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.128056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a01b-d760-4ede-97c1-83f1a8c7d36b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:34.138786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:34.139387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.139710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.139979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] PATH_INFO: `/v3/projects/67568d58a68e4837a58c5432a01ca9fa` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.145088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67568d58a68e4837a58c5432a01ca9fa)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:34.146918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d97cd6-ab31-4ae8-971f-847edf1fcf25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:34.148976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:34.157100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d97cd6-ab31-4ae8-971f-847edf1fcf25 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:34.197891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 823/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:34.266371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67568d58a68e4837a58c5432a01ca9fa', 'request_id': 'req-f192b6d9-7985-42ce-8064-62f302c2b239'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:34.276779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67568d58a68e4837a58c5432a01ca9fa', 'request_id': 'req-f192b6d9-7985-42ce-8064-62f302c2b239'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:34.302046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67568d58a68e4837a58c5432a01ca9fa', 'request_id': 'req-f192b6d9-7985-42ce-8064-62f302c2b239'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:34.314228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:34.318936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] The token cache is being invalidate because project 67568d58a68e4837a58c5432a01ca9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:34.319368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:34.320347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f192b6d9-7985-42ce-8064-62f302c2b239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:34.328209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 821/1644] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:34 2022] DELETE /identity/v3/projects/67568d58a68e4837a58c5432a01ca9fa => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:34.329772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.330129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.330486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.373911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:34.376899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:34.478743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 824/1645] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:34.486502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.487529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.487613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.538280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:34.542047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:34.553123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 822/1646] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:34.554648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:34.567264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f14813-bfab-4ab5-bb1f-ac05bad060e8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:34.578025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:34.579454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.579454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.579891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.581224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:34.582199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:34.587039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:34.631677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:34.636518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 825/1647] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:34.651384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8478c9-c15a-4415-a99c-e363a4fdb6f5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:34.669831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:34.670572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.671424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.671894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.672672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:34.678302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:34.730772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 823/1648] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:34 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:44:34.735543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:34.753664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9ad2c-7644-495f-9838-bfef04a3c15b admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:34.865852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:34.866832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.867662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.868144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.871224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:34.881477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:34.945422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:34.952400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 826/1649] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:34 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:34.961097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0acce-cc60-4889-95f3-90d282ade0d7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:34.971946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:34.972905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:34.973288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:34.973997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:34.976733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:34.978559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:35.010030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 824/1650] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:35.014562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:35.029404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cb38a-4f95-4075-ac81-30b2bbe86fc1 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:35.040109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad217e87-a82e-44b3-a72a-da8744cdde23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:35.041217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217e87-a82e-44b3-a72a-da8744cdde23 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.041703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217e87-a82e-44b3-a72a-da8744cdde23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.043425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217e87-a82e-44b3-a72a-da8744cdde23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.046062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217e87-a82e-44b3-a72a-da8744cdde23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:35.048019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217e87-a82e-44b3-a72a-da8744cdde23 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:35.068215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 827/1651] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:35.072389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:35.089487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cbdfc-c923-4e11-8de2-29bf0216ec61 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:35.095879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:35.097524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.097919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.098640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] PATH_INFO: `/v3/projects/8338fa77b1244a9bbc25191d2c04a8c1/users/be54397075144b2d90c375cbf5d3d9b9/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.134033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8338fa77b1244a9bbc25191d2c04a8c1, user_id=be54397075144b2d90c375cbf5d3d9b9, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:35.137441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:35.165321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 825/1652] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:35 2022] PUT /identity/v3/projects/8338fa77b1244a9bbc25191d2c04a8c1/users/be54397075144b2d90c375cbf5d3d9b9/roles/87f1831257824560813f954348534540 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:35.178005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.178544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.178784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.260353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] MFA Rules not processed for user `be54397075144b2d90c375cbf5d3d9b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:35.262491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:35.383744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:35.437971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:35.440669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:35.441343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 828/1653] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:35.445534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:35.585799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd341-ec06-459e-9369-4ffdd2a6fe53 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:35.603110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:35.604361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.604698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.604982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.605647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:35.609867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:35.614889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472b1cf-fe76-456e-b2fd-4ebc9ad31e78 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:35.628257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:35.629191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.629643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.630195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] PATH_INFO: `/v3/projects/d0742191cd4d45d0a67eaf82b377e2c3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.631921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0742191cd4d45d0a67eaf82b377e2c3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:35.644929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:35.671782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 829/1654] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:35 2022] POST /identity/v3/projects => generated 369 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:44:35.676578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:35.747055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0742191cd4d45d0a67eaf82b377e2c3', 'request_id': 'req-10de74b8-ea41-4b9b-b43f-e95470c80098'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:35.765650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0742191cd4d45d0a67eaf82b377e2c3', 'request_id': 'req-10de74b8-ea41-4b9b-b43f-e95470c80098'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:35.797584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0742191cd4d45d0a67eaf82b377e2c3', 'request_id': 'req-10de74b8-ea41-4b9b-b43f-e95470c80098'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:35.798529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50ec6c-d3c6-4365-a45e-2db321c34a4f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:35.824071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] The token cache is being invalidate because project d0742191cd4d45d0a67eaf82b377e2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:35.824438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:35.825195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10de74b8-ea41-4b9b-b43f-e95470c80098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:35.826938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 826/1655] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:35 2022] DELETE /identity/v3/projects/d0742191cd4d45d0a67eaf82b377e2c3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:35.831746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274a15-2b26-438e-940e-33697949a2e2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.832099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274a15-2b26-438e-940e-33697949a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.832446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274a15-2b26-438e-940e-33697949a2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.881417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f0274a15-2b26-438e-940e-33697949a2e2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:35.882891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f0274a15-2b26-438e-940e-33697949a2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:35.918037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:35.918737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:35.919190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:35.919495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:35.920209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:35.923994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:35.942257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:35.971203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 830/1656] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:35 2022] POST /identity/v3/users => generated 291 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:44:35.974923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:35.977385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:36.000197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e42a4-417c-4e89-be57-a22ab8f3b3a4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.008808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.010121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.010544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.010956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.012119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.034715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.038443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:36.046551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 827/1657] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:36.054954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.055190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.055951 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.073807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 831/1658] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:44:36.074761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.099898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de2d7d-b9b7-454e-bc33-2a733db963fd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.108140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:36.110455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:36.112737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.113433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.113714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.113984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] PATH_INFO: `/v3/projects/669493e38b6c4892bab38e43030bbb5a/users/a115218626bd4f2db0fe29bffda22718/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.137363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 828/1659] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:36.141197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.150849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1b3bf-a307-4453-a6f3-109f04fd5ef6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.153511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] RBAC: Authorizing `identity:create_grant(project_id=669493e38b6c4892bab38e43030bbb5a, user_id=a115218626bd4f2db0fe29bffda22718, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.156739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.171932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.172644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.172995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.173352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.174082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:36.175090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.176843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.181468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 832/1660] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:36 2022] PUT /identity/v3/projects/669493e38b6c4892bab38e43030bbb5a/users/a115218626bd4f2db0fe29bffda22718/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:36.185548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.215913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 829/1661] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:36.216947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.238860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c6d28-354d-4852-8694-828fca32cd56 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.339739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e72b4e-94cc-4429-9129-d8586c98522a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.353223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.353787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.354065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.355154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.355872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.359993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.387549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 833/1662] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:44:36.390277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.391752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.392388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.394732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.395103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.395701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.401543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.414639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a61b7-9978-46a5-bf10-cd26da49c41f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.428358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fbe3-4813-4109-9313-ce458690c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.429049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fbe3-4813-4109-9313-ce458690c303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.429707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fbe3-4813-4109-9313-ce458690c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.430017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fbe3-4813-4109-9313-ce458690c303 admin admin] PATH_INFO: `/v3/projects/669493e38b6c4892bab38e43030bbb5a/users/a115218626bd4f2db0fe29bffda22718/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.436018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787fbe3-4813-4109-9313-ce458690c303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=669493e38b6c4892bab38e43030bbb5a, user_id=a115218626bd4f2db0fe29bffda22718, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.439331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787fbe3-4813-4109-9313-ce458690c303 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.459057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 830/1663] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:36 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:36.462967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.480040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 834/1664] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:36 2022] PUT /identity/v3/projects/669493e38b6c4892bab38e43030bbb5a/users/a115218626bd4f2db0fe29bffda22718/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:36.484742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.485034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.485553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.548504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] MFA Rules not processed for user `a115218626bd4f2db0fe29bffda22718`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:36.550901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:36.631936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea320e-e476-4cae-bc10-c4f7ac39019d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.662671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.664524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.666960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.681773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.686116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.689698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.735064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 835/1665] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:36.746423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.748276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:36.753602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 831/1666] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:36 2022] POST /identity/v3/users => generated 286 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:36.754034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.792756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb77a8-560a-49cb-ab6c-755ab2804fa0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.805796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.806602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.806888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.807157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.807960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.809727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.822015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:36.856583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 832/1667] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:44:36.862825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50e837-ffe7-4bac-a592-932739e9a325 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.867739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.868847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.869720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.870846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.871319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.886469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28e21-cb13-4479-89fb-c652a55a5e79 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.888892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.890843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.901674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.902300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.902580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.902854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:36.903567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:36.905207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:36.905647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 836/1668] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:36.916578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:36.923842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 833/1669] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:36.926846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:36.964204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e86bc-9612-465d-8860-d6b30f7b8fb0 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:36.980360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b266b-68a0-4a0e-a6c3-969adf5c378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:36.982475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b266b-68a0-4a0e-a6c3-969adf5c378a admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:36.987557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b266b-68a0-4a0e-a6c3-969adf5c378a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:36.988262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b266b-68a0-4a0e-a6c3-969adf5c378a admin admin] PATH_INFO: `/v3/projects/ed48010356f246aabdedc3f2855d2f00/users/529ed8ad430545b996e5e227cabff52c/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.014312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b266b-68a0-4a0e-a6c3-969adf5c378a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed48010356f246aabdedc3f2855d2f00, user_id=529ed8ad430545b996e5e227cabff52c, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:37.016532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.017405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b266b-68a0-4a0e-a6c3-969adf5c378a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:37.019187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:37.040396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdead01-b81e-4b52-aa85-e6d587b1d6a2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:37.046159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 837/1670] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:36 2022] PUT /identity/v3/projects/ed48010356f246aabdedc3f2855d2f00/users/529ed8ad430545b996e5e227cabff52c/roles/87f1831257824560813f954348534540 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:37.057625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.058056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.058425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.129503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] MFA Rules not processed for user `529ed8ad430545b996e5e227cabff52c`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:37.130802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:37.181069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:37.184068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.184370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.184647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.197795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:37.235658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 838/1671] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:37.241870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.324255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659df93-27e3-400e-b5fd-861c75c04cc0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:37.329864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:37.332927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.333084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.333350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.334207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:37.336182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:37.390594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 839/1672] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:37 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:37.392322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.432706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:37.438759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f62c8b-a079-455a-af6a-ea0549c5e1ec service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:37.441703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:37.482502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 834/1673] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:37.482947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1b0b-3e0a-4858-b97d-64dafcbb4cdc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:37.490466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:37.491409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.491719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.491992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.493662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:37.497503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:37.521457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.521742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.522016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.544314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 840/1674] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:37 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:37.551096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.582171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6f075-3aee-4e87-b518-f8d5255c4546 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:37.590878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] MFA Rules not processed for user `c5307118357e44ffbff5074ae4a94c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:37.592420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:37.597830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:37.598464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.598735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.599010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.600045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:37.605809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:37.635673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 841/1675] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:37.637140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.659975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9dfe9-ae7f-45fa-b401-7056eea48d6e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:37.686155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:37.686836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:37.687114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:37.687405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] PATH_INFO: `/v3/projects/4507ea6a38b24c26ba4ce61c2abc0970/users/36bffc3bf8504742b9e007e7ef7f5ce0/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:37.739077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4507ea6a38b24c26ba4ce61c2abc0970, user_id=36bffc3bf8504742b9e007e7ef7f5ce0, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:37.745222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:37.787538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 842/1676] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:37 2022] PUT /identity/v3/projects/4507ea6a38b24c26ba4ce61c2abc0970/users/36bffc3bf8504742b9e007e7ef7f5ce0/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:37.792253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.813855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 835/1677] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:37.839643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:37.841096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:37.999101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:38.008114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4603a62-ce15-4fcd-836c-89303e38aaba admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:38.021458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:38.022553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:38.022834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:38.023108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:38.023805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:38.025427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:38.075741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9abd6-1ba9-47ff-af9f-bc39441446b3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:38.078664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 843/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:38.082834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:38.102884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7fc8e-c4e4-4b88-a366-ed6af69abcfb admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:38.116932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc3ab6-2333-4958-af5c-5f290f11bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:38.117710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc3ab6-2333-4958-af5c-5f290f11bb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:38.117879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc3ab6-2333-4958-af5c-5f290f11bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:38.118155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc3ab6-2333-4958-af5c-5f290f11bb4f admin admin] PATH_INFO: `/v3/projects/4507ea6a38b24c26ba4ce61c2abc0970/users/36bffc3bf8504742b9e007e7ef7f5ce0/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:38.121746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc3ab6-2333-4958-af5c-5f290f11bb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4507ea6a38b24c26ba4ce61c2abc0970, user_id=36bffc3bf8504742b9e007e7ef7f5ce0, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:38.125581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc3ab6-2333-4958-af5c-5f290f11bb4f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:38.167841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 844/1679] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:38 2022] PUT /identity/v3/projects/4507ea6a38b24c26ba4ce61c2abc0970/users/36bffc3bf8504742b9e007e7ef7f5ce0/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:38.179294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:38.180064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:38.180561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:38.295761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:38.296974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:38.297352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:38.297718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:38.298771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] MFA Rules not processed for user `36bffc3bf8504742b9e007e7ef7f5ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:38.309814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:38.398963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:38.403933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:38.416574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 836/1680] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:38.462257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:38.501501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 845/1681] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:38.514457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:38.690334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1515f-1fea-49fc-8629-bc4835d1b8a7 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:38.745748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:38.787991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417676fa-8104-4100-be3e-473aa16525c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:38.788587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417676fa-8104-4100-be3e-473aa16525c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:38.788868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417676fa-8104-4100-be3e-473aa16525c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:38.789282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417676fa-8104-4100-be3e-473aa16525c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:38.804642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417676fa-8104-4100-be3e-473aa16525c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:38.808170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417676fa-8104-4100-be3e-473aa16525c2 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:38.829468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 837/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:38.997574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:39.015537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c783a2-9bb7-40d6-bc87-4d204b244cc0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:39.040029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e89bd5-077a-473c-98fe-c7312fdb6125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:39.041446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e89bd5-077a-473c-98fe-c7312fdb6125 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:39.042940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e89bd5-077a-473c-98fe-c7312fdb6125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:39.043413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e89bd5-077a-473c-98fe-c7312fdb6125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:39.278620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e89bd5-077a-473c-98fe-c7312fdb6125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:39.282917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e89bd5-077a-473c-98fe-c7312fdb6125 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:39.318001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 846/1683] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:41.740320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96247e9b-6024-445b-9b02-d47e91b9824e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:41.741304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96247e9b-6024-445b-9b02-d47e91b9824e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:41.741888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96247e9b-6024-445b-9b02-d47e91b9824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:41.828180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-96247e9b-6024-445b-9b02-d47e91b9824e None None] MFA Rules not processed for user `4e0e6a33679f4cac96704f71dcfa5c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:41.829643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-96247e9b-6024-445b-9b02-d47e91b9824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:42.051587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 838/1684] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:42.057558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.058075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.058170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.104908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] MFA Rules not processed for user `4e0e6a33679f4cac96704f71dcfa5c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:42.109991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:42.135165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 847/1685] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:42.205978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.206270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.206551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.227783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:42.271255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] MFA Rules not processed for user `c5307118357e44ffbff5074ae4a94c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:42.273096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:42.463760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0250fb-c3d7-4b50-ad13-e504f79ab287 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:42.469545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 839/1686] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:42.474066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:42.477073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:42.479599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8033383-7efc-4a21-b910-0891f52d1197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:42.480234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8033383-7efc-4a21-b910-0891f52d1197 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.480515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8033383-7efc-4a21-b910-0891f52d1197 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.480787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8033383-7efc-4a21-b910-0891f52d1197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.501195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8033383-7efc-4a21-b910-0891f52d1197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:42.504097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8033383-7efc-4a21-b910-0891f52d1197 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:42.530196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 848/1687] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:42.600441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8be89-291b-48a3-829c-4fcf841f5406 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.600750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8be89-291b-48a3-829c-4fcf841f5406 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.601056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8be89-291b-48a3-829c-4fcf841f5406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.614987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa03289-dc56-4b2f-b286-8a72a1be80a0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:42.627696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:42.628355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.628809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.629093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] PATH_INFO: `/v3/users/5cf5ef2e256f426ca72574e40caf1ecd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.630412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf5ef2e256f426ca72574e40caf1ecd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:42.636172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:42.679168 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:42.689983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bec8be89-291b-48a3-829c-4fcf841f5406 None None] MFA Rules not processed for user `33070b58c99741eeb9f97a73507d012e`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:42.691627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bec8be89-291b-48a3-829c-4fcf841f5406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:42.796005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf5ef2e256f426ca72574e40caf1ecd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:42.817562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf5ef2e256f426ca72574e40caf1ecd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:42.819946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf5ef2e256f426ca72574e40caf1ecd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:42.822990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf5ef2e256f426ca72574e40caf1ecd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:42.849447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8130a280-6204-4410-b861-537a34d0b4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf5ef2e256f426ca72574e40caf1ecd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:42.856862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 849/1688] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:42.867161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.867592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.868340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.892266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:42.922113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 840/1689] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:42 2022] DELETE /identity/v3/users/5cf5ef2e256f426ca72574e40caf1ecd => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:42.927394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e3e5-7680-4673-b860-e4cebba104d6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:42.932314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e3e5-7680-4673-b860-e4cebba104d6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:42.932598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e3e5-7680-4673-b860-e4cebba104d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:42.940687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] MFA Rules not processed for user `33070b58c99741eeb9f97a73507d012e`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:42.942099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:42.988165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3225e3e5-7680-4673-b860-e4cebba104d6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:43.009753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3225e3e5-7680-4673-b860-e4cebba104d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:43.100527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 850/1690] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:43.133221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 841/1691] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:43.184319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:43.188489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:43.337989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243bc75-7967-4b08-9b98-8ef7a4c82388 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.338281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243bc75-7967-4b08-9b98-8ef7a4c82388 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.338559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243bc75-7967-4b08-9b98-8ef7a4c82388 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.340631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 842/1692] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 00:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:43.347579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30c540-882a-4b70-b5b0-12ca2690c62f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.347899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b3ece-52ff-465b-ad27-48931081ceed None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:43.348003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30c540-882a-4b70-b5b0-12ca2690c62f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.348277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30c540-882a-4b70-b5b0-12ca2690c62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.352209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:43.352898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.353184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.353456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.396437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:43.403361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:43.427852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 851/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:43.444334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ea30c540-882a-4b70-b5b0-12ca2690c62f None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:43.445850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ea30c540-882a-4b70-b5b0-12ca2690c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:43.468322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 843/1694] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:44:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:43.469027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:43.471204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:43.491599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fef3b-7ca7-4465-897e-468e09bf449e service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:43.501432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b6064-72b2-4e67-b98f-4045d7d240ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:43.503614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b6064-72b2-4e67-b98f-4045d7d240ef service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.504800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b6064-72b2-4e67-b98f-4045d7d240ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.505180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b6064-72b2-4e67-b98f-4045d7d240ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.523816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b6064-72b2-4e67-b98f-4045d7d240ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:43.525890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b6064-72b2-4e67-b98f-4045d7d240ef service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:43.548239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 852/1695] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:43.665512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f16d6-bfdf-4b29-8fce-1809edd9b8ed None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.666072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f16d6-bfdf-4b29-8fce-1809edd9b8ed None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.666767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f16d6-bfdf-4b29-8fce-1809edd9b8ed None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.685747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 844/1696] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 00:44:43 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:43.696841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.697384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.697963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.812978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:43.817394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:43.820635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f16d6-bfdf-4b29-8fce-1809edd9b8ed None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:43.822537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-916f16d6-bfdf-4b29-8fce-1809edd9b8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:43.834759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 853/1697] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:44:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:43.858359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f16d6-bfdf-4b29-8fce-1809edd9b8ed None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:43.873112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:43.874955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.875396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.876143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] PATH_INFO: `/v3/users/be54397075144b2d90c375cbf5d3d9b9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.878374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=be54397075144b2d90c375cbf5d3d9b9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:43.883755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:43.894958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:43.898771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1895cb-4ce9-4a94-8382-80d2a5ada681 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:43.915979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8165b-fcbe-4b63-86eb-f18a2823d109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:43.918328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8165b-fcbe-4b63-86eb-f18a2823d109 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:43.918328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8165b-fcbe-4b63-86eb-f18a2823d109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:43.920348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8165b-fcbe-4b63-86eb-f18a2823d109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:43.948108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8165b-fcbe-4b63-86eb-f18a2823d109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:43.953188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8165b-fcbe-4b63-86eb-f18a2823d109 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:43.966524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 854/1698] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:44.034638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be54397075144b2d90c375cbf5d3d9b9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.055967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be54397075144b2d90c375cbf5d3d9b9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.056832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be54397075144b2d90c375cbf5d3d9b9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.058074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be54397075144b2d90c375cbf5d3d9b9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.070355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be54397075144b2d90c375cbf5d3d9b9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.075372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:44.096506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 845/1699] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:43 2022] DELETE /identity/v3/users/be54397075144b2d90c375cbf5d3d9b9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:44:44.103666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:44.104142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:44.104212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:44.139432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:44.144565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:44.191321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:44.193260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:44.347221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 855/1700] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:44.364400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:44.366680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:44.404800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5be80-b10c-4504-8002-f5999292aa2f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:44.411253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:44.420302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:44.421169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:44.421570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:44.422612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] PATH_INFO: `/v3/projects/b77ee72d79e048be9cb68bf296ec6bec` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:44.432169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77ee72d79e048be9cb68bf296ec6bec)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:44.440659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:44.549949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce45b64-3a79-409b-8cb2-49c23e8e9d3a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:44.585005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672fd4a-b741-4b25-a83d-8385163698b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:44.585736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672fd4a-b741-4b25-a83d-8385163698b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:44.587794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672fd4a-b741-4b25-a83d-8385163698b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:44.592964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672fd4a-b741-4b25-a83d-8385163698b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:44.624013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672fd4a-b741-4b25-a83d-8385163698b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:44.628709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672fd4a-b741-4b25-a83d-8385163698b6 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:44.634178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77ee72d79e048be9cb68bf296ec6bec', 'request_id': 'req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.652847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77ee72d79e048be9cb68bf296ec6bec', 'request_id': 'req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.654063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 856/1701] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:44.687643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77ee72d79e048be9cb68bf296ec6bec', 'request_id': 'req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.714538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] The token cache is being invalidate because project b77ee72d79e048be9cb68bf296ec6bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:44.714835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.715678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:44.717795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 846/1702] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:44 2022] DELETE /identity/v3/projects/b77ee72d79e048be9cb68bf296ec6bec => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:44:44.723243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:44.723243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:44.723584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:44.771266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:44.773119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:44.848012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:44.849513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:44.988116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 857/1703] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:44.989824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:44.991054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.023280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad991a3-9f40-4417-9cb5-e8192fb6c7c8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:45.046270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:45.047142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.047550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.047929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.078561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.121482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:45.125284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cdfda-e4f7-40a5-8910-c57293b20c5b service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:45.154120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 847/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:45.157145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc273-7e14-4ae1-80d6-24ae2ff42105 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.157440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc273-7e14-4ae1-80d6-24ae2ff42105 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.157719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc273-7e14-4ae1-80d6-24ae2ff42105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.179578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856bc86-7172-40cc-b8fb-91298b5cea3e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:45.186797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:45.191845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:45.192592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.193101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.193521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.201221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.225288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1e1fc273-7e14-4ae1-80d6-24ae2ff42105 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:45.228593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1e1fc273-7e14-4ae1-80d6-24ae2ff42105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.268946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 848/1705] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:45.270351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3338ab-7544-461d-8d78-26c5a8841bf1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.270641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3338ab-7544-461d-8d78-26c5a8841bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.270919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3338ab-7544-461d-8d78-26c5a8841bf1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.286457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 849/1706] 10.222.0.15 () {54 vars in 961 bytes} [Thu Sep 15 00:44:45 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:45.287659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3338ab-7544-461d-8d78-26c5a8841bf1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:45.322286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3338ab-7544-461d-8d78-26c5a8841bf1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:45.332904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:45.334574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.334758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.335041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.336214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:45.336527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:45.338180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e8b48-b99e-48e3-8f19-d76c46d1868f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:45.379971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 850/1707] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:45.384441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:45.385491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.386182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28270cd-e8a0-4378-a241-809275ac8852 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:45.392839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.393321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.397218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:45.430128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 858/1708] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:45.435386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.435845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.441193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.477524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:45.482962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.516779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 851/1709] 10.222.0.15 () {58 vars in 1058 bytes} [Thu Sep 15 00:44:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:45.518667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:45.523703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] MFA Rules not processed for user `4e0e6a33679f4cac96704f71dcfa5c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:45.525190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.545725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae32c6-b41a-4b52-b3c3-4d765be1dabf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:45.560748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:45.561476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.561763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.562045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.562770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:45.565237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:45.629828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 852/1710] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:45 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:45.632847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:45.634735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:45.706657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 859/1711] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:45.706657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:45.730467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75797a6-6d70-432a-99f2-b18af75f7eb0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:45.844620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0cc18-4bde-4822-a37b-d27e8e69dfeb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:45.865706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:45.866471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:45.866771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:45.867049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] PATH_INFO: `/v3/projects/8338fa77b1244a9bbc25191d2c04a8c1` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:45.877444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8338fa77b1244a9bbc25191d2c04a8c1)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:45.883357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.010842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:46.011499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.014959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.015251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.045380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:46.049596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.073435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 860/1712] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:46.074291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:46.092082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8338fa77b1244a9bbc25191d2c04a8c1', 'request_id': 'req-8a0f15b0-bfbc-434c-988a-320a3865c39e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.107972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5c28-b33f-49de-b39a-a27b81c8368d service placement] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:46.122312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8338fa77b1244a9bbc25191d2c04a8c1', 'request_id': 'req-8a0f15b0-bfbc-434c-988a-320a3865c39e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.142019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8338fa77b1244a9bbc25191d2c04a8c1', 'request_id': 'req-8a0f15b0-bfbc-434c-988a-320a3865c39e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.197089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] The token cache is being invalidate because project 8338fa77b1244a9bbc25191d2c04a8c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:46.209973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.209973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.209973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 853/1713] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:45 2022] DELETE /identity/v3/projects/8338fa77b1244a9bbc25191d2c04a8c1 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:46.229062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:46.232528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:46.303016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:46.303715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.303999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.304277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.304928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:46.306856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.358175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 861/1714] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:46 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:46.361932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:46.363702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:46.391700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4599d-62db-4528-9ef4-50ca7f0ab9d0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:46.421271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:46.421907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.422186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.425260 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.426693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:46.428373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.433114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:46.454420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 862/1715] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:44:46.460553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:46.482910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f15b0-bfbc-434c-988a-320a3865c39e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:46.489694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926eccb-810e-4b28-9490-0aed2976471e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:46.496699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:46.497757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.497757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.498045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] PATH_INFO: `/v3/users/a115218626bd4f2db0fe29bffda22718` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.499371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a115218626bd4f2db0fe29bffda22718)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:46.501380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.519360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:46.520266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.525270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.525734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.527792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:46.535415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.552129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:46.580166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 863/1716] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:46.585893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:46.621110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a115218626bd4f2db0fe29bffda22718'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.631169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bb44c-303d-4f95-9766-2e8e9d0cac40 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:46.638815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a115218626bd4f2db0fe29bffda22718'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.640901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a115218626bd4f2db0fe29bffda22718'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.641946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a115218626bd4f2db0fe29bffda22718'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.651269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbf9903e-0b5d-4c1b-927c-d33af996c217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a115218626bd4f2db0fe29bffda22718'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:46.662463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:46.663143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.663413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.663705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] PATH_INFO: `/v3/projects/83747b69e79d43eca9616f0c2661d016/users/7790e72f5caa4055b8a8eb8fce643743/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.685426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 854/1717] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:46 2022] DELETE /identity/v3/users/a115218626bd4f2db0fe29bffda22718 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:46.712952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83747b69e79d43eca9616f0c2661d016, user_id=7790e72f5caa4055b8a8eb8fce643743, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:46.716508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:46.745295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 864/1718] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:46 2022] PUT /identity/v3/projects/83747b69e79d43eca9616f0c2661d016/users/7790e72f5caa4055b8a8eb8fce643743/roles/87f1831257824560813f954348534540 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:46.759365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:46.759923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:46.760100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:46.849497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] MFA Rules not processed for user `7790e72f5caa4055b8a8eb8fce643743`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:46.852457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:47.106266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 855/1719] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:47.114404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:47.116640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:47.351410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0656c79-7e86-43fd-9287-18b278c923d8 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:47.378534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:47.379404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:47.379846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:47.380283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:47.381326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:47.383767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:47.447548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:47.461729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:47.502555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce67d0-a90c-4a57-8cad-16ba264a382f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:47.523303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 865/1720] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:47 2022] POST /identity/v3/projects => generated 387 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:47.528070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:47.530316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:47.531604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:47.531965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:47.532194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] PATH_INFO: `/v3/projects/669493e38b6c4892bab38e43030bbb5a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:47.534254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669493e38b6c4892bab38e43030bbb5a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:47.536295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:47.718233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669493e38b6c4892bab38e43030bbb5a', 'request_id': 'req-9f6d1212-efe0-47fe-a695-061249db9c39'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:47.719642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b08e6-897a-4807-8336-0422608980ee admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:47.720105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:47.734965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669493e38b6c4892bab38e43030bbb5a', 'request_id': 'req-9f6d1212-efe0-47fe-a695-061249db9c39'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:47.754221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669493e38b6c4892bab38e43030bbb5a', 'request_id': 'req-9f6d1212-efe0-47fe-a695-061249db9c39'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:47.773197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] The token cache is being invalidate because project 669493e38b6c4892bab38e43030bbb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:47.773687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:47.777704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f6d1212-efe0-47fe-a695-061249db9c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:47.782879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 856/1721] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:47 2022] DELETE /identity/v3/projects/669493e38b6c4892bab38e43030bbb5a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:47.789369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625c26-d0a3-4a8e-b9d0-e3cacddcf4ad None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:47.789967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625c26-d0a3-4a8e-b9d0-e3cacddcf4ad None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:47.790340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625c26-d0a3-4a8e-b9d0-e3cacddcf4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:47.851637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2e625c26-d0a3-4a8e-b9d0-e3cacddcf4ad None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:47.854287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2e625c26-d0a3-4a8e-b9d0-e3cacddcf4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:47.931151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:47.933044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:47.936886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:47.937946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:47.938917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:47.941469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.028318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:48.042056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 857/1722] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:48.051994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.052512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.053010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.065748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 866/1723] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:47 2022] POST /identity/v3/users => generated 300 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:44:48.067650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.085155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:48.113045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8f98a-4a75-408d-9bde-b6b9ac4026b6 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.132547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.134704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.135949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.139167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.140868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:48.145626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.147420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:48.162780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:48.170111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 867/1724] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:48.171209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.186550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023e2f-7ca3-4187-b957-6ce3a4ddc7e9 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.207028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.208348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.210572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.212071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] PATH_INFO: `/v3/projects/3fd9e41390c0492e80181c64811d7794/users/8a017d4c2c26408ebcf67df891877692/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.216737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 858/1725] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:48.218313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.250854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7c212-09d1-4dda-a8ed-69423e3792fc None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.258151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.258891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.259849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.260378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.261411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:48.262468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:48.265383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.279818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd9e41390c0492e80181c64811d7794, user_id=8a017d4c2c26408ebcf67df891877692, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:48.282864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 859/1726] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:48.285473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.300266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.330761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12975fe1-fc29-46e7-9ca3-4a4cf3fa7fcd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.365303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 868/1727] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:48 2022] PUT /identity/v3/projects/3fd9e41390c0492e80181c64811d7794/users/8a017d4c2c26408ebcf67df891877692/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:48.371024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.503877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.505230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.505522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.505797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.509086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:48.514986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.552124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888527b6-142a-4c00-8f0b-2e8047965384 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.580444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.581094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.581380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.581672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.582471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:48.590151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.641396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 869/1728] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:48.646203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 860/1729] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:48 2022] POST /identity/v3/projects => generated 387 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:48.646621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.653775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:48.698613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a44887-771f-40dd-9768-a64a3fbfc0eb admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.891502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:48.949402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15167-a85c-4952-b768-c8487fec0bff admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:48.963269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.964588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.966268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.966874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:48.968865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:48.972306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:48.981797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc586d-b5cf-49e3-9746-ff6f49843c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:48.994908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc586d-b5cf-49e3-9746-ff6f49843c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:48.996095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc586d-b5cf-49e3-9746-ff6f49843c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:48.998266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc586d-b5cf-49e3-9746-ff6f49843c87 admin admin] PATH_INFO: `/v3/projects/3fd9e41390c0492e80181c64811d7794/users/8a017d4c2c26408ebcf67df891877692/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.004345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc586d-b5cf-49e3-9746-ff6f49843c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd9e41390c0492e80181c64811d7794, user_id=8a017d4c2c26408ebcf67df891877692, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:49.008519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc586d-b5cf-49e3-9746-ff6f49843c87 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:49.025144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 861/1730] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:48 2022] POST /identity/v3/users => generated 294 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:49.035064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:49.049104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b483bf-74ce-4d22-8872-25b29ee8c6ca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:49.066919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 870/1731] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:48 2022] PUT /identity/v3/projects/3fd9e41390c0492e80181c64811d7794/users/8a017d4c2c26408ebcf67df891877692/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:49.082900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.083432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.083950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.219897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] MFA Rules not processed for user `8a017d4c2c26408ebcf67df891877692`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:49.222823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:49.223436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:49.225902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.227582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.227911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.228719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:49.233824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:49.282223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 862/1732] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:49.283652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:49.312922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333e3ed-781e-431b-8605-b5a9647a8485 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:49.326107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:49.328565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.329291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.330635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.333361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:49.335128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:49.346520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:49.376290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 863/1733] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:49.381601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:49.405090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7d86-feaa-40c6-83c1-cc41a6adf9ed admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:49.414837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0c5ff-e044-49d3-a463-45429a7b50f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:49.415567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0c5ff-e044-49d3-a463-45429a7b50f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.415852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0c5ff-e044-49d3-a463-45429a7b50f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.416125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0c5ff-e044-49d3-a463-45429a7b50f6 admin admin] PATH_INFO: `/v3/projects/50862e13563d4a6a9ca6ee91a09c2423/users/73b0838dcfb2437088a406dcd89850ba/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.445187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 871/1734] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:49.468086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:49.469593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:49.470814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0c5ff-e044-49d3-a463-45429a7b50f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50862e13563d4a6a9ca6ee91a09c2423, user_id=73b0838dcfb2437088a406dcd89850ba, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:49.475037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0c5ff-e044-49d3-a463-45429a7b50f6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:49.499959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 864/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:49 2022] PUT /identity/v3/projects/50862e13563d4a6a9ca6ee91a09c2423/users/73b0838dcfb2437088a406dcd89850ba/roles/87f1831257824560813f954348534540 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:49.514694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.514694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.514694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.588008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] MFA Rules not processed for user `73b0838dcfb2437088a406dcd89850ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:49.591465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:49.598921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86382f-9e6d-4107-934e-232f8f0de701 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:49.731740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:49.732216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.732499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.733876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.799583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 865/1736] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:49.804641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:49.887435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:49.945364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:49.950109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:49.979702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675b7f9-2cf3-449c-82c6-cc4f9927c126 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:49.984682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:49.984879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 872/1737] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:49.985496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:49.986086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:49.986086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:49.986700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:49.988962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:50.009212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:50.034053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ef34c-8525-4eca-a0e3-8b463052f643 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:50.038269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 866/1738] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:49 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:44:50.042235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:50.050874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:50.052127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.052424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.052699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] PATH_INFO: `/v3/users/7790e72f5caa4055b8a8eb8fce643743` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.056520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7790e72f5caa4055b8a8eb8fce643743)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:50.062778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:50.175819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:50.186587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2eda1-1c08-413c-9457-8275e72765a8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:50.203305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:50.203856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.204191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.204506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.205847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:50.210697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7790e72f5caa4055b8a8eb8fce643743'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:50.210785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:50.265315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7790e72f5caa4055b8a8eb8fce643743'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:50.266126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7790e72f5caa4055b8a8eb8fce643743'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:50.277853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 867/1739] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:50 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:50.280796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7790e72f5caa4055b8a8eb8fce643743'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:50.282058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-38fd73e9-1d67-4b3c-95c6-6c3957de31c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7790e72f5caa4055b8a8eb8fce643743'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:50.283061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:50.285539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:50.315412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5eb71-75a5-4e7c-b9d0-258aead5b334 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:50.337748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:50.338511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.338798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.339075 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.339779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:50.350136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 873/1740] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:50 2022] DELETE /identity/v3/users/7790e72f5caa4055b8a8eb8fce643743 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:44:50.355905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:50.372328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec206b9b-3260-4e56-8daa-3cf762328cc9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.372581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec206b9b-3260-4e56-8daa-3cf762328cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.382971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec206b9b-3260-4e56-8daa-3cf762328cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.403916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 868/1741] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:44:50.406931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:50.466531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df6810-fd94-4c24-b18b-230f9ecb1815 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:50.468116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ec206b9b-3260-4e56-8daa-3cf762328cc9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:50.472491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ec206b9b-3260-4e56-8daa-3cf762328cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:50.605691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 874/1742] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:50.609368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ac04d-69bf-42dc-9ec5-2757d6265f2f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.610624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ac04d-69bf-42dc-9ec5-2757d6265f2f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.610914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ac04d-69bf-42dc-9ec5-2757d6265f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.717361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:50.718567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.718921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.720603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.722335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:50.724961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:50.751850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 869/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:50.751850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:50.761188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:50.834826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:50.856819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-793ac04d-69bf-42dc-9ec5-2757d6265f2f None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:50.863220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-793ac04d-69bf-42dc-9ec5-2757d6265f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:50.906185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ef48-dccc-4055-9cea-1c1ee1d37c5c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:50.915111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:50.915961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:50.916452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:50.916909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:50.937343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:50.939035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:50.997134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 870/1744] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:50.998094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:51.015919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4fb66-9b05-4af2-8bc6-b29ebc092b5d service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:51.036414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:51.036763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:51.037123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:51.037396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] PATH_INFO: `/v3/projects/7b4f3ae4d36b46b5bb21e2eadfdc396f/users/10ddbe2b00e640d8aadc85d085618214/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:51.052782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 875/1745] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:44:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:51.082046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4f3ae4d36b46b5bb21e2eadfdc396f, user_id=10ddbe2b00e640d8aadc85d085618214, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:51.085160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:51.112260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 871/1746] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:50 2022] PUT /identity/v3/projects/7b4f3ae4d36b46b5bb21e2eadfdc396f/users/10ddbe2b00e640d8aadc85d085618214/roles/87f1831257824560813f954348534540 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:51.118183 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:51.118469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:51.118750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:51.184925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] MFA Rules not processed for user `10ddbe2b00e640d8aadc85d085618214`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:51.186194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:51.216929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:51.218706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:51.258286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:51.453593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deacc08-8a2c-4cb0-980b-c9ff1ada2f68 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:51.461834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 876/1747] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:51.461834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:51.464656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:51.469545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:51.471173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:51.471722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:51.472907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] PATH_INFO: `/v3/users/ebda414ffd59490d86758dca0342d353` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:51.474305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebda414ffd59490d86758dca0342d353)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:51.476765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:51.626051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebda414ffd59490d86758dca0342d353'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:51.648560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebda414ffd59490d86758dca0342d353'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:51.649300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebda414ffd59490d86758dca0342d353'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:51.650072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebda414ffd59490d86758dca0342d353'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:51.664747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebda414ffd59490d86758dca0342d353'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:51.701715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 872/1748] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:51 2022] DELETE /identity/v3/users/ebda414ffd59490d86758dca0342d353 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:51.702632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:51.706639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:51.754674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:51.834883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5007eb-ab00-4416-8b80-815ed8bbc026 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:51.897327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620291c3-2d75-4df2-b52f-94cdebcce178 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:51.903145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5285f-7004-4e47-9de0-49f14330228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:51.903922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5285f-7004-4e47-9de0-49f14330228d admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:51.904319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5285f-7004-4e47-9de0-49f14330228d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:51.904689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5285f-7004-4e47-9de0-49f14330228d admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:51.905557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f5285f-7004-4e47-9de0-49f14330228d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:51.909073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f5285f-7004-4e47-9de0-49f14330228d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:51.966444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 873/1749] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:51 2022] POST /identity/v3/projects => generated 387 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:51.970150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:51.970829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:51.971073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:51.971272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:51.971398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:51.971786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:51.973022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:51.974055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:52.026649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:52.030359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:52.083612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:52.153396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:52.157792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:52.176300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 877/1750] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:44:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:44:52.177208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:52.196662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5a24-4575-4f7e-90cd-dafa445877be service swift] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:52.324706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 874/1751] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:52.371061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:52.376505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:52.445418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:52.459706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:52.460112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:52.460420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:52.461135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:52.463188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:52.585486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 878/1752] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:52 2022] POST /identity/v3/users => generated 300 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:52.592772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:52.654054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5fab-813c-4eb3-b766-e51825cd9979 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:52.682970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:52.684167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76201e5-5309-43d9-a82f-e1a964f2f649 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:52.687897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:52.688185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:52.688473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:52.689193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:52.703067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:52.705953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:52.711812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:52.711994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:52.713775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:52.761052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:52.761702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 879/1753] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:52.764898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:52.766298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:52.813042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f03b13-b8ce-4dc5-b71b-ea116fd5e4af admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:52.816179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 875/1754] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:52.840211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:52.843170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:52.843493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:52.843845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] PATH_INFO: `/v3/projects/8e58edc84dd1478fb8c339bca269229a/users/30406093669446d98af75f3dee7bb12b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:52.903452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e58edc84dd1478fb8c339bca269229a, user_id=30406093669446d98af75f3dee7bb12b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:52.912540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:52.962464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:52.963050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 880/1755] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:52 2022] PUT /identity/v3/projects/8e58edc84dd1478fb8c339bca269229a/users/30406093669446d98af75f3dee7bb12b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:52.976341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9699a29-06e8-404b-8025-42b4a008c961 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:53.222781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:53.247976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656ae0-3dc4-46fc-9cf2-28321c8d35aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:53.249446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656ae0-3dc4-46fc-9cf2-28321c8d35aa admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:53.249952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656ae0-3dc4-46fc-9cf2-28321c8d35aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:53.250565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656ae0-3dc4-46fc-9cf2-28321c8d35aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:53.251379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78656ae0-3dc4-46fc-9cf2-28321c8d35aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:53.255293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78656ae0-3dc4-46fc-9cf2-28321c8d35aa admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:53.295207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:53.305519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 876/1756] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:53.311528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88a80a-df4c-4ffd-b510-63a2097b5b0d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:53.319999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:53.320902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:53.321447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:53.321867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] PATH_INFO: `/v3/projects/8e58edc84dd1478fb8c339bca269229a/users/30406093669446d98af75f3dee7bb12b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:53.324898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e58edc84dd1478fb8c339bca269229a, user_id=30406093669446d98af75f3dee7bb12b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:53.328942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:53.357037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58f17d-9ca5-4db8-80b0-c06b1fe3b26d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:53.357352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58f17d-9ca5-4db8-80b0-c06b1fe3b26d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:53.357631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58f17d-9ca5-4db8-80b0-c06b1fe3b26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:53.360737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 881/1757] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:53 2022] PUT /identity/v3/projects/8e58edc84dd1478fb8c339bca269229a/users/30406093669446d98af75f3dee7bb12b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:53.399141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8d58f17d-9ca5-4db8-80b0-c06b1fe3b26d None None] MFA Rules not processed for user `30406093669446d98af75f3dee7bb12b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:53.402059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8d58f17d-9ca5-4db8-80b0-c06b1fe3b26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:53.605038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:53.606622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 877/1758] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:53.607467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:53.656042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:53.656330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:53.656628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:53.704664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:53.714657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:53.742538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe6b0c-648a-44e9-a9e2-866bd4f60682 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:53.753736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:53.754539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:53.754924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:53.755374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:53.779235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:53.780841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:53.813620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 882/1759] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:44:53.898583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 878/1760] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Sep 15 00:44:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:54.014584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:54.016941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:54.074420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62635-d075-47ad-92b1-503b91e0829d service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:54.082440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:54.084975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:54.086945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:54.087293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:54.117717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:54.120495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:54.152068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 883/1761] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:54.668081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:54.685897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:54.936642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f4eea-aaf4-4363-b651-797ad43e5e52 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:54.953250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:54.954003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:54.954281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:54.954576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] PATH_INFO: `/v3/projects/83747b69e79d43eca9616f0c2661d016` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:54.955909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83747b69e79d43eca9616f0c2661d016)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:54.961924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:55.032326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:55.091323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83747b69e79d43eca9616f0c2661d016', 'request_id': 'req-a0dc5017-a656-49b6-a465-d0edccce2a33'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.099616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83747b69e79d43eca9616f0c2661d016', 'request_id': 'req-a0dc5017-a656-49b6-a465-d0edccce2a33'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.118079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83747b69e79d43eca9616f0c2661d016', 'request_id': 'req-a0dc5017-a656-49b6-a465-d0edccce2a33'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.182975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] The token cache is being invalidate because project 83747b69e79d43eca9616f0c2661d016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:55.185390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.186389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.190146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 879/1762] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:54 2022] DELETE /identity/v3/projects/83747b69e79d43eca9616f0c2661d016 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:55.194122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:55.223129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a082dc3-59f8-4780-9a79-a294be5c2256 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:55.246351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:55.383673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:55.384365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:55.384637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:55.385097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] PATH_INFO: `/v3/users/8a017d4c2c26408ebcf67df891877692` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:55.386231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a017d4c2c26408ebcf67df891877692)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:55.388730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:55.483726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:55.485382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:55.527445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a017d4c2c26408ebcf67df891877692'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.541839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a017d4c2c26408ebcf67df891877692'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.542892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a017d4c2c26408ebcf67df891877692'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.553441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a017d4c2c26408ebcf67df891877692'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.557800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a017d4c2c26408ebcf67df891877692'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:55.594697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 884/1763] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:44:55 2022] DELETE /identity/v3/users/8a017d4c2c26408ebcf67df891877692 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:55.723510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5017-a656-49b6-a465-d0edccce2a33 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:55.876788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:55.877799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:55.878238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:55.878686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:55.885056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:55.979844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:55.981472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:56.075647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:56.076936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35c224-e1d8-4779-accb-eb97bd48f842 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:56.120574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 880/1764] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:56.201424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33966cfd-dd50-48bc-8277-93a326b7e24a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:56.215162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:56.215651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:56.216159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:56.216578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] PATH_INFO: `/v3/projects/09628995c5034e75b393d75f0a537404` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:56.222531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09628995c5034e75b393d75f0a537404)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:56.223669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: WARNING py.warnings [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 00:44:56.223669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: warnings.warn(msg) Sep 15 00:44:56.223669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: Sep 15 00:44:56.225124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:56.377468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:56.429869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09628995c5034e75b393d75f0a537404', 'request_id': 'req-bb5a2865-8561-455a-a733-fecc63310734'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:56.448111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09628995c5034e75b393d75f0a537404', 'request_id': 'req-bb5a2865-8561-455a-a733-fecc63310734'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:56.471959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09628995c5034e75b393d75f0a537404', 'request_id': 'req-bb5a2865-8561-455a-a733-fecc63310734'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:56.524385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] The token cache is being invalidate because project 09628995c5034e75b393d75f0a537404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:56.524701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:56.533730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bb5a2865-8561-455a-a733-fecc63310734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:56.545779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 885/1765] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:55 2022] DELETE /identity/v3/projects/09628995c5034e75b393d75f0a537404 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:44:56.547918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:56.548212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:56.549519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:56.596491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:56.597747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:56.649205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:56.778379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 881/1766] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:56.781304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:56.781767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:56.782187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:56.861069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:56.864100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:56.891976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 886/1767] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:56.896712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:56.909015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2c56b-4653-4180-97ce-6061b9ae49a9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:56.935310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:56.941305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:56.941977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:56.947853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:56.948287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:56.948701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:56.951398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:56.995948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 882/1768] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:56.996149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:57.017556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c45968-11a1-4e7d-8e24-5926e942960e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:57.026295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:57.027337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:57.029989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:57.033761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:57.035022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:57.038811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:57.094863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 887/1769] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:56 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:44:57.100686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:57.118233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1567e8-a8ff-4e1c-8035-de56e0b12fb4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:57.228727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:57.231226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:57.232829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:57.232829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:57.233714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:57.240533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:57.330671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 883/1770] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:57 2022] POST /identity/v3/users => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:57.335142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:57.378224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc3f6a-2b95-4e05-8cfa-4087b620d422 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:57.412881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:57.413812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:57.414304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:57.414773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:57.416749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:57.425567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:57.450124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 888/1771] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:57.455107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:57.468786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62d14-a016-46bd-bd73-94ee1e642ab4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:57.485787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e27ba-4d62-4602-9a15-b69fd9c3627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:57.488120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e27ba-4d62-4602-9a15-b69fd9c3627f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:57.489214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e27ba-4d62-4602-9a15-b69fd9c3627f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:57.490726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e27ba-4d62-4602-9a15-b69fd9c3627f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:57.493987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e27ba-4d62-4602-9a15-b69fd9c3627f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:57.500097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e27ba-4d62-4602-9a15-b69fd9c3627f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:57.529924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 884/1772] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:57.532100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:57.552866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d299fe1-44ab-427c-856a-17393022e7cd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:57.585921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:57.587999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:57.588252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:57.588534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] PATH_INFO: `/v3/projects/417bf94ed53447b9927e8a20535464ab/users/dd777be73a7d446ab40a3eb730ccdb49/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:57.632473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417bf94ed53447b9927e8a20535464ab, user_id=dd777be73a7d446ab40a3eb730ccdb49, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:57.643517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:57.676758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 889/1773] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:57 2022] PUT /identity/v3/projects/417bf94ed53447b9927e8a20535464ab/users/dd777be73a7d446ab40a3eb730ccdb49/roles/87f1831257824560813f954348534540 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:57.681606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:57.682678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:57.683060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:57.744403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:57.745728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:58.048152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 885/1774] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:44:58.106919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:58.109114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:58.262982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cbc36-f460-4115-a7c1-5d5c52f5b0f2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:58.289420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df6e6d-2d39-4b10-996a-81e1672b8fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:58.290137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6e6d-2d39-4b10-996a-81e1672b8fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:58.291882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6e6d-2d39-4b10-996a-81e1672b8fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:58.292227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6e6d-2d39-4b10-996a-81e1672b8fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:58.313064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6e6d-2d39-4b10-996a-81e1672b8fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:58.315710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6e6d-2d39-4b10-996a-81e1672b8fc1 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:58.330441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 890/1775] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:58.448356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:58.450153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:58.475281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e073-c524-4de0-ba2c-4f3cec75af7e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:58.500837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:58.502681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:58.502973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:58.504692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] PATH_INFO: `/v3/projects/3fd9e41390c0492e80181c64811d7794` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:58.509605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd9e41390c0492e80181c64811d7794)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:58.519298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:58.631273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:58.683258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd9e41390c0492e80181c64811d7794', 'request_id': 'req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:58.705060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd9e41390c0492e80181c64811d7794', 'request_id': 'req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:58.717934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd9e41390c0492e80181c64811d7794', 'request_id': 'req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:58.758739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] The token cache is being invalidate because project 3fd9e41390c0492e80181c64811d7794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:44:58.759043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:58.764456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b4495ec1-4c5a-4bba-aa8d-5143885d2a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:44:58.770363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 886/1776] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:44:58 2022] DELETE /identity/v3/projects/3fd9e41390c0492e80181c64811d7794 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:58.771845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87706d-42c2-46ac-8d71-255812e0cc6b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:58.772331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87706d-42c2-46ac-8d71-255812e0cc6b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:58.772632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87706d-42c2-46ac-8d71-255812e0cc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:58.816251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2f87706d-42c2-46ac-8d71-255812e0cc6b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:58.818740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2f87706d-42c2-46ac-8d71-255812e0cc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:58.883163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7efa-32df-4f5c-b0b0-6e47ef07f3d2 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:58.883738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7efa-32df-4f5c-b0b0-6e47ef07f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:58.884166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7efa-32df-4f5c-b0b0-6e47ef07f3d2 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:58.886950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 887/1777] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:44:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:44:58.927664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 891/1778] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:58.934710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:58.934973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:58.935274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:58.987357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:58.996809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:58.997596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:58.998062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.008807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:59.022014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:44:59.052300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 888/1779] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:59.054919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.059878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:59.061246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:59.094054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef65b0-4e08-49a2-a0dd-2d8bc8d098ce None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.100767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.101669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.101991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.102526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.103503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:44:59.104042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.106207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.137726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 889/1780] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:44:59.143284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.168379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d145-f461-465f-86f6-6351dec50999 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.180202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.181263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.186111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.186781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.187991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.190504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.258283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 892/1781] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:44:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:59.338951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.340560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:59.358620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 890/1782] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:44:59 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:59.359941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.387409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e2031-bc2b-4ec8-8c15-974a3e0b1e3d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.522266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfc6cb-da6f-44cd-9ac2-f29e938e6bc4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.537184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39655d55-4f7e-4f2b-b0a7-fcbba1f1eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.539450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39655d55-4f7e-4f2b-b0a7-fcbba1f1eb43 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.540318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39655d55-4f7e-4f2b-b0a7-fcbba1f1eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.540740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39655d55-4f7e-4f2b-b0a7-fcbba1f1eb43 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.542236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39655d55-4f7e-4f2b-b0a7-fcbba1f1eb43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.544777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39655d55-4f7e-4f2b-b0a7-fcbba1f1eb43 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.601617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 891/1783] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:44:59 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:44:59.603239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50a777-d2af-4309-b9e5-e013c0346390 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.604529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50a777-d2af-4309-b9e5-e013c0346390 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.606604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50a777-d2af-4309-b9e5-e013c0346390 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.608158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 892/1784] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:44:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:44:59.610177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50a777-d2af-4309-b9e5-e013c0346390 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.666267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50a777-d2af-4309-b9e5-e013c0346390 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.666456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39490126-a2b9-43c3-9282-6118280dc859 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.666925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39490126-a2b9-43c3-9282-6118280dc859 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.690446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39490126-a2b9-43c3-9282-6118280dc859 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.690727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39490126-a2b9-43c3-9282-6118280dc859 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.694348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.696076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.708209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.708741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.709702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.711760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.717450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39490126-a2b9-43c3-9282-6118280dc859 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.719087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39490126-a2b9-43c3-9282-6118280dc859 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.747941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 893/1785] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:59.750814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.778526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 893/1786] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:44:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:44:59.790486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab72f89-1811-40a6-b9dc-e363c4017d01 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.807000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.810709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66e17b-8daf-4bbd-b4dc-ef3f170b08b4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.811216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66e17b-8daf-4bbd-b4dc-ef3f170b08b4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.811328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.811438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66e17b-8daf-4bbd-b4dc-ef3f170b08b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.811636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.813655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.814457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.817438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.845176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 894/1787] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:44:59.847692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:44:59.869070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175de51-bbef-4726-9e1a-e0a90480dc62 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:44:59.879164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4c66e17b-8daf-4bbd-b4dc-ef3f170b08b4 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:44:59.880488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4c66e17b-8daf-4bbd-b4dc-ef3f170b08b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:44:59.883592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:44:59.883592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.883592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.883906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] PATH_INFO: `/v3/projects/9cb7c27e52414d8b873e28395caa60c2/users/1c706dee12094e778b8b386e910ab16c/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:44:59.915110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 894/1788] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:44:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:44:59.919195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb7c27e52414d8b873e28395caa60c2, user_id=1c706dee12094e778b8b386e910ab16c, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:44:59.927503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:44:59.952869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 895/1789] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:44:59 2022] PUT /identity/v3/projects/9cb7c27e52414d8b873e28395caa60c2/users/1c706dee12094e778b8b386e910ab16c/roles/87f1831257824560813f954348534540 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:44:59.969359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:44:59.969639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:44:59.970181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:00.030791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:00.057925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:00.060099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] MFA Rules not processed for user `1c706dee12094e778b8b386e910ab16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:00.061603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:00.202500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b423d-7d7f-4261-94c8-871216388009 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:00.215816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:00.216615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:00.217462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:00.217462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:00.241013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:00.243444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:00.293532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 896/1790] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:00.335752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 895/1791] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:44:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:00.336913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:00.497827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90833f39-a156-4322-8f85-768754c2d78d service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:00.512289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:00.515753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:00.516054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:00.516336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:00.520230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:00.524556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:00.602762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 897/1792] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:00 2022] POST /identity/v3/projects => generated 381 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:00.607044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:00.613599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:00.635125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0538e-9742-419b-a96a-27acc9b78c8d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:00.807748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:00.811358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:00.826222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:00.827892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:00.829100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:00.829732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:00.833561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:00.841430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:00.844452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2c00b-a9c0-40b5-8e6e-4406af1f1aee admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:00.857053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:00.859516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:00.860131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:00.862211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] PATH_INFO: `/v3/users/eff3aa87f49348e2ad73c520810bc88f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:00.866920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff3aa87f49348e2ad73c520810bc88f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:00.872801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:00.909438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 896/1793] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:00 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:00.913017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:00.968035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b88dd-e6a9-4836-a678-3d26897a1142 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:00.992146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:00.993640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:00.994237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:00.995537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:00.997351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:01.009160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:01.038143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 897/1794] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:01.040961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:01.041698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff3aa87f49348e2ad73c520810bc88f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:01.056495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff3aa87f49348e2ad73c520810bc88f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:01.057247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff3aa87f49348e2ad73c520810bc88f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:01.057939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff3aa87f49348e2ad73c520810bc88f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:01.061779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e7e56-aba1-4848-b42b-bcf2df6458d3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:01.072472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:01.073389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:01.073766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:01.074160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] PATH_INFO: `/v3/projects/2a6f5bf591664ba59d150eac32c6ee87/users/009dd0fce1fa4b82bf38e576128ecb68/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:01.074771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2dc97174-9385-43ff-830b-d09306aae587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff3aa87f49348e2ad73c520810bc88f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:01.110721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:01.115045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6f5bf591664ba59d150eac32c6ee87, user_id=009dd0fce1fa4b82bf38e576128ecb68, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:01.125675 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:01.126857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 898/1795] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:00 2022] DELETE /identity/v3/users/eff3aa87f49348e2ad73c520810bc88f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:01.132237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:01.132911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:01.134860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:01.152992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 898/1796] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:01 2022] PUT /identity/v3/projects/2a6f5bf591664ba59d150eac32c6ee87/users/009dd0fce1fa4b82bf38e576128ecb68/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:01.162094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:01.165302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:01.183709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:01.188288 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:01.396707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62138386-03e1-4f22-8a90-f949a956d405 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:01.413588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:01.414228 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:01.415032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:01.415032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:01.416038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:01.425213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:01.452130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 899/1797] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:01.454945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 899/1798] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:01.461993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:01.482819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed710851-0551-4e00-b88b-9962543dc86e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:01.503209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dee336-aca4-4934-87a2-4e786cb47185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:01.505348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dee336-aca4-4934-87a2-4e786cb47185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:01.505697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dee336-aca4-4934-87a2-4e786cb47185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:01.507642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dee336-aca4-4934-87a2-4e786cb47185 admin admin] PATH_INFO: `/v3/projects/2a6f5bf591664ba59d150eac32c6ee87/users/009dd0fce1fa4b82bf38e576128ecb68/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:01.513700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dee336-aca4-4934-87a2-4e786cb47185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6f5bf591664ba59d150eac32c6ee87, user_id=009dd0fce1fa4b82bf38e576128ecb68, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:01.521086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dee336-aca4-4934-87a2-4e786cb47185 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:01.527177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:01.529006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:01.556919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 900/1799] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:01 2022] PUT /identity/v3/projects/2a6f5bf591664ba59d150eac32c6ee87/users/009dd0fce1fa4b82bf38e576128ecb68/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:01.568043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:01.568287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:01.568453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:01.658488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] MFA Rules not processed for user `009dd0fce1fa4b82bf38e576128ecb68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:01.659931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:01.693790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aa93b-1de0-4d9b-a7fc-ba85627eec9e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:01.706308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:01.814959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:01.815803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:01.816103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:01.816357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:01.911815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:01.913361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:01.930715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 900/1800] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:01.944919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 901/1801] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:01.959038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:01.962083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:02.148976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6550-7fbc-4084-a7d9-d8eb78d80013 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:02.158894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:02.159554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:02.159842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:02.160119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:02.187310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:02.191130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:02.242119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 901/1802] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:02.243874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:02.245303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:02.266150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0fe56-512b-44ac-8904-ecaed5cd5391 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:02.273440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:02.274376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:02.274793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:02.275004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:02.279804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:02.471031 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:02.476102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313690c-2ec4-41c8-b29d-e8c809e70d7d service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:02.498200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 902/1803] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:03.664381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:03.690838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa146c21-2e38-4644-b4a9-ebeeb526e2b2 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:03.703713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:03.705542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:03.705834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:03.707346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] PATH_INFO: `/v3/users/1c706dee12094e778b8b386e910ab16c` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:03.709284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c706dee12094e778b8b386e910ab16c)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:03.716513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:03.830484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c706dee12094e778b8b386e910ab16c'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:03.861357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c706dee12094e778b8b386e910ab16c'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:03.864069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c706dee12094e778b8b386e910ab16c'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:03.875564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:03.896537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c706dee12094e778b8b386e910ab16c'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:03.898728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c706dee12094e778b8b386e910ab16c'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:03.959363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 902/1804] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:03 2022] DELETE /identity/v3/users/1c706dee12094e778b8b386e910ab16c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:03.964152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:03.964691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:03.964691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:03.992264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:04.000385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:04.002139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:04.119621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 903/1805] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:04.149340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:04.151389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:04.262891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0664-1e96-495e-b77c-07c8699cc905 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:04.267567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e07ab-22e3-4c2b-afcb-188eff999ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:04.268196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e07ab-22e3-4c2b-afcb-188eff999ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:04.268491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e07ab-22e3-4c2b-afcb-188eff999ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:04.268756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e07ab-22e3-4c2b-afcb-188eff999ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:04.290701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e07ab-22e3-4c2b-afcb-188eff999ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:04.293204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e07ab-22e3-4c2b-afcb-188eff999ba3 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:04.316413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 903/1806] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:04.763315 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:04.765406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:04.809610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704f756-e2b1-4f6d-b744-fc3a24462d78 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:04.827359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:04.828086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:04.828358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:04.828621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] PATH_INFO: `/v3/projects/fe33d8cbe3e04bdeb9c9691e048ba925` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:04.830355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe33d8cbe3e04bdeb9c9691e048ba925)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:04.832488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:04.883633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:04.884151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:04.884246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:04.964188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] MFA Rules not processed for user `dd777be73a7d446ab40a3eb730ccdb49`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:04.968450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:04.972482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe33d8cbe3e04bdeb9c9691e048ba925', 'request_id': 'req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.024196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe33d8cbe3e04bdeb9c9691e048ba925', 'request_id': 'req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.091789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe33d8cbe3e04bdeb9c9691e048ba925', 'request_id': 'req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.120693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] The token cache is being invalidate because project fe33d8cbe3e04bdeb9c9691e048ba925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:05.121221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.125332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.129221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 904/1807] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:04 2022] DELETE /identity/v3/projects/fe33d8cbe3e04bdeb9c9691e048ba925 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:05.134052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.139888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:05.147499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 904/1808] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:05.151778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.153342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:05.320884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d47ab-f585-4639-875b-cf1a23820543 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.334726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78bc5c-fdfb-4a0d-b94d-81157d9be49e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.335733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:05.336384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:05.336789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:05.336900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:05.337538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:05.361592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:05.365069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:05.366016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:05.366518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:05.367082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] PATH_INFO: `/v3/users/c1783b7aa9994de5b7f94a7702b0f355` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:05.368547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1783b7aa9994de5b7f94a7702b0f355)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:05.371053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:05.394426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:05.429392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 905/1809] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:05 2022] POST /identity/v3/projects => generated 405 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:05.430402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.432923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:05.433453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:05.449190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1783b7aa9994de5b7f94a7702b0f355'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.473293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1783b7aa9994de5b7f94a7702b0f355'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.473919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1783b7aa9994de5b7f94a7702b0f355'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.474671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1783b7aa9994de5b7f94a7702b0f355'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.484345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1783b7aa9994de5b7f94a7702b0f355'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.508858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 905/1810] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:05 2022] DELETE /identity/v3/users/c1783b7aa9994de5b7f94a7702b0f355 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:05.513411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.514780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:05.577052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820bee-7849-4874-9c7d-efec840bfc40 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.639238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47071c7c-7114-4a59-8c41-108643886a3b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.645143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:05.646038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:05.646491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:05.646959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:05.647796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:05.652314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:05.684365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:05.689649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:05.690581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:05.691080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] PATH_INFO: `/v3/projects/9cb7c27e52414d8b873e28395caa60c2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:05.693113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb7c27e52414d8b873e28395caa60c2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:05.696537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 906/1811] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:05 2022] POST /identity/v3/users => generated 309 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:05.699757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:05.701373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.714652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0710a12-86db-4e53-9059-2257f6298410 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.722762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:05.723968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:05.724579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:05.724902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:05.727331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:05.729391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:05.747687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 907/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:05.749517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.768079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a2fdb-546a-4f74-bb10-bccb83cdee3a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.772595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb7c27e52414d8b873e28395caa60c2', 'request_id': 'req-b659f879-d970-45bd-ba9e-580aba83b664'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.780010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb7c27e52414d8b873e28395caa60c2', 'request_id': 'req-b659f879-d970-45bd-ba9e-580aba83b664'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.795839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb7c27e52414d8b873e28395caa60c2', 'request_id': 'req-b659f879-d970-45bd-ba9e-580aba83b664'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.819292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] The token cache is being invalidate because project 9cb7c27e52414d8b873e28395caa60c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:05.819632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.820315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:05.822254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 906/1813] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:05 2022] DELETE /identity/v3/projects/9cb7c27e52414d8b873e28395caa60c2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:05.826513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.828386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:05.868860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:05.875240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:05.876783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:05.877516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:05.877794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] PATH_INFO: `/v3/projects/1ede4436931542f4a61b528bb8a939c9/users/46823c0ddbe14f1792be2fab7b931306/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:05.900939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ede4436931542f4a61b528bb8a939c9, user_id=46823c0ddbe14f1792be2fab7b931306, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:05.903693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:05.926432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 908/1814] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:05 2022] PUT /identity/v3/projects/1ede4436931542f4a61b528bb8a939c9/users/46823c0ddbe14f1792be2fab7b931306/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:05.927687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:05.927833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659f879-d970-45bd-ba9e-580aba83b664 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:05.929489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:05.959305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:06.012327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.013749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.014410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.014767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] PATH_INFO: `/v3/users/009dd0fce1fa4b82bf38e576128ecb68` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.016343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=009dd0fce1fa4b82bf38e576128ecb68)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.019460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.043549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05aa64-671e-43b1-ba75-d10c6493aad9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.051345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.055638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.057926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.058236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.059157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.061724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.098832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 909/1815] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:06.100142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.104157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.116187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009dd0fce1fa4b82bf38e576128ecb68'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.121678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2564a6-de6b-4897-a2b8-aa710850bd80 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.129161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.130113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.130714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.131273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] PATH_INFO: `/v3/projects/031df15ccfd44744a2bc495b971b1330` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.132701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031df15ccfd44744a2bc495b971b1330)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.135363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009dd0fce1fa4b82bf38e576128ecb68'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.135507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.137378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009dd0fce1fa4b82bf38e576128ecb68'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.155969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009dd0fce1fa4b82bf38e576128ecb68'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.156955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009dd0fce1fa4b82bf38e576128ecb68'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.189262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 907/1816] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:05 2022] DELETE /identity/v3/users/009dd0fce1fa4b82bf38e576128ecb68 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:06.190229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.194107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.253752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031df15ccfd44744a2bc495b971b1330', 'request_id': 'req-68c81399-8321-4f38-8a94-e7b1c53cbe04'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.275325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031df15ccfd44744a2bc495b971b1330', 'request_id': 'req-68c81399-8321-4f38-8a94-e7b1c53cbe04'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.291642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031df15ccfd44744a2bc495b971b1330', 'request_id': 'req-68c81399-8321-4f38-8a94-e7b1c53cbe04'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.294532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:06.308630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442de7a-0130-4113-a7ba-f55ffffb37dd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.320774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] The token cache is being invalidate because project 031df15ccfd44744a2bc495b971b1330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:06.321073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.322705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-68c81399-8321-4f38-8a94-e7b1c53cbe04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.339959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 910/1817] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:06 2022] DELETE /identity/v3/projects/031df15ccfd44744a2bc495b971b1330 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:06.339959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1a3c4-cbac-4112-8033-71479e0e0671 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.339959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1a3c4-cbac-4112-8033-71479e0e0671 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.339959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1a3c4-cbac-4112-8033-71479e0e0671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.387697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a7a1a3c4-cbac-4112-8033-71479e0e0671 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:06.389994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a7a1a3c4-cbac-4112-8033-71479e0e0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.428387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.429518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.430773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.431204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] PATH_INFO: `/v3/projects/1ede4436931542f4a61b528bb8a939c9/users/46823c0ddbe14f1792be2fab7b931306/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.433417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ede4436931542f4a61b528bb8a939c9, user_id=46823c0ddbe14f1792be2fab7b931306, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.438281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.459007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 908/1818] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:06 2022] PUT /identity/v3/projects/1ede4436931542f4a61b528bb8a939c9/users/46823c0ddbe14f1792be2fab7b931306/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:06.464641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.466806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.508689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 911/1819] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:06.518036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.518589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.518883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.542526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:06.543778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.581404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:06.589623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947d929-96fa-44db-994d-2ce2dd699a7c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.596799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.597688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.597810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.598085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] PATH_INFO: `/v3/users/73b0838dcfb2437088a406dcd89850ba` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.599458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73b0838dcfb2437088a406dcd89850ba)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.602971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.664586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 912/1820] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:06.670318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.683654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df2e7c-57e2-446b-adc1-554a882fbf1b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.691708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.693795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.694113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.694699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.697371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:06.698274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.700836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73b0838dcfb2437088a406dcd89850ba'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.701925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.722004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73b0838dcfb2437088a406dcd89850ba'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.723526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73b0838dcfb2437088a406dcd89850ba'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.739707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 913/1821] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:06.739896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73b0838dcfb2437088a406dcd89850ba'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.739896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73b0838dcfb2437088a406dcd89850ba'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.745359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.747379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.765636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 909/1822] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:06 2022] DELETE /identity/v3/users/73b0838dcfb2437088a406dcd89850ba => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:06.767565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.769336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f227-6bbc-4c1d-a0a1-618818bcc22f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.769467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.795138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:06.849492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b4ce6-b37c-4145-8a12-fc0606d54a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.850255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b4ce6-b37c-4145-8a12-fc0606d54a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.850556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b4ce6-b37c-4145-8a12-fc0606d54a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.850904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b4ce6-b37c-4145-8a12-fc0606d54a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.851660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b4ce6-b37c-4145-8a12-fc0606d54a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.854342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b4ce6-b37c-4145-8a12-fc0606d54a55 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.862104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb45c5-41a9-4ba4-b564-1b4b3fc101d7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:06.869462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:06.870532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.870811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.871513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] PATH_INFO: `/v3/projects/2a6f5bf591664ba59d150eac32c6ee87` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.872326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6f5bf591664ba59d150eac32c6ee87)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:06.875782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:06.889155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 914/1823] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:06 2022] POST /identity/v3/projects => generated 401 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:06.889956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a047-fd88-4c29-b381-200aec2da100 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.890316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a047-fd88-4c29-b381-200aec2da100 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.890663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a047-fd88-4c29-b381-200aec2da100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:06.911022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7e17a047-fd88-4c29-b381-200aec2da100 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:06.912371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7e17a047-fd88-4c29-b381-200aec2da100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.939802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6f5bf591664ba59d150eac32c6ee87', 'request_id': 'req-256113b6-7e6e-446a-9bcf-c765f8bdc875'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.947319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6f5bf591664ba59d150eac32c6ee87', 'request_id': 'req-256113b6-7e6e-446a-9bcf-c765f8bdc875'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.958439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6f5bf591664ba59d150eac32c6ee87', 'request_id': 'req-256113b6-7e6e-446a-9bcf-c765f8bdc875'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.972223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] The token cache is being invalidate because project 2a6f5bf591664ba59d150eac32c6ee87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:06.972702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.973488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:06.975385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 910/1824] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:06 2022] DELETE /identity/v3/projects/2a6f5bf591664ba59d150eac32c6ee87 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:06.976618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:06.978226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:06.988606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 915/1825] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:06.990643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:06.991011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:06.991377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.014485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:07.015284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:07.064366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256113b6-7e6e-446a-9bcf-c765f8bdc875 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.069335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.070141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.070514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.070876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.071842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.073847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.081608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:07.097312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 911/1826] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:06 2022] POST /identity/v3/users => generated 301 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:07.098154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.099894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:07.109398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 916/1827] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:07.112364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.128075 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ef017-e8b2-46ff-8641-4b9a0c55c1e7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.133377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e08ed-0137-4094-b1d7-ebbef8f356fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.134022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e08ed-0137-4094-b1d7-ebbef8f356fc admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.134307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e08ed-0137-4094-b1d7-ebbef8f356fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.134588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e08ed-0137-4094-b1d7-ebbef8f356fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.135244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e08ed-0137-4094-b1d7-ebbef8f356fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.136879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e08ed-0137-4094-b1d7-ebbef8f356fc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.148114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 917/1828] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:07.148114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.148114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.148114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.159524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:07.170562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:07.171764 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:07.181763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 918/1829] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:07.183093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.185569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d14ee-92cb-4d90-b430-ef67b4e7680a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.191061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.192475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.192862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.193392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.195775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aacb32-dc3c-4020-a971-5ce482afd50e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.204758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.205368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.205670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.205955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.206604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.208277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.215177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.218895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 919/1830] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:07.219148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.223516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.232625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 912/1831] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:07.233706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.244170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb5f99-3968-4bee-922a-4ac3c1235d3f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.245652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8551a-f790-42e2-935f-a0f985af01e0 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.250466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.251641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.251751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572deafd-82d1-4c8c-93f7-38992028cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.251961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.252203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.252437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572deafd-82d1-4c8c-93f7-38992028cc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.252665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572deafd-82d1-4c8c-93f7-38992028cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.252849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:07.252971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572deafd-82d1-4c8c-93f7-38992028cc96 admin admin] PATH_INFO: `/v3/projects/7ed32bd8cf964c67be3d5b02dcbd86c2/users/05c6b9c83002419198d4fc106cd5c509/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.253140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.255586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.270645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 920/1832] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:07.275407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.279110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572deafd-82d1-4c8c-93f7-38992028cc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed32bd8cf964c67be3d5b02dcbd86c2, user_id=05c6b9c83002419198d4fc106cd5c509, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.282781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572deafd-82d1-4c8c-93f7-38992028cc96 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.286393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe98e9-70eb-4f34-b887-0d256fc8962f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.294516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.295084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.295360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.295701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.296991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.300064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.303565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 913/1833] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:07 2022] PUT /identity/v3/projects/7ed32bd8cf964c67be3d5b02dcbd86c2/users/05c6b9c83002419198d4fc106cd5c509/roles/87f1831257824560813f954348534540 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:07.314107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.315385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.315757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.346318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 921/1834] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:07.351114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.361592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] MFA Rules not processed for user `05c6b9c83002419198d4fc106cd5c509`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:07.362881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:07.441254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e456d2-6e4e-440c-9f37-d7be25ea6e00 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.446861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.447505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.447781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.448057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.448690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.450631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.459461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 914/1835] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:07.463906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.475875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739fc55-1329-4a95-a3dd-4b7aa0f84149 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.476820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 922/1836] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:07.480583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.482166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.482783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.483064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.483335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.484443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.488109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.494280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8524-b7af-44b2-8d00-c0c4026362a1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.504285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.505197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.505536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.506317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.507041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.508906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.521918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 923/1837] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:07.528631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.529428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 915/1838] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/projects => generated 401 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:07.533819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.543096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641ddf8-314f-4f31-9616-c87c43674534 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.606589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21489-4b8c-49c3-95e3-fa0a8ae9d83f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.612798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.613337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.613612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.613885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.615199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.617403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.634162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.635227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.635699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.636161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.637060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.639402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.641400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:07.666109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 924/1839] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:07.670421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.672222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 916/1840] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/users => generated 307 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:07.676627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.689851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fb75c-63f3-4b63-b90e-68c39aa8ccfd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.695368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166224d9-91a5-43f9-9483-e2fd04b365bf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.695696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.696659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.696991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.697324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.698029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.699859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.700299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394c2e7-ffc2-4f69-a253-b6d53397fb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.700973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394c2e7-ffc2-4f69-a253-b6d53397fb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.701254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394c2e7-ffc2-4f69-a253-b6d53397fb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.701530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394c2e7-ffc2-4f69-a253-b6d53397fb3f admin admin] PATH_INFO: `/v3/projects/7eec17aa13734b8a9eca85b9b8161a5d/users/b9fe5383178d4164a71e258eba1c06ba/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.707690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:07.717984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 917/1841] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:07.720225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.721624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394c2e7-ffc2-4f69-a253-b6d53397fb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eec17aa13734b8a9eca85b9b8161a5d, user_id=b9fe5383178d4164a71e258eba1c06ba, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.724842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394c2e7-ffc2-4f69-a253-b6d53397fb3f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.732769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3408d-08d1-4663-af86-8799a661ae1e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.742296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.743313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.743857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.744367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] PATH_INFO: `/v3/projects/714ff544a3db484abf71c9f97f2bc0e2/users/ab638d0d58134112a96eb72a9904eadb/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.771803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 925/1842] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:07 2022] PUT /identity/v3/projects/7eec17aa13734b8a9eca85b9b8161a5d/users/b9fe5383178d4164a71e258eba1c06ba/roles/87f1831257824560813f954348534540 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:07.772089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=714ff544a3db484abf71c9f97f2bc0e2, user_id=ab638d0d58134112a96eb72a9904eadb, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.777056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.777372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.777953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.778502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.800870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 918/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:07 2022] PUT /identity/v3/projects/714ff544a3db484abf71c9f97f2bc0e2/users/ab638d0d58134112a96eb72a9904eadb/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:07.804937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.832225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] MFA Rules not processed for user `b9fe5383178d4164a71e258eba1c06ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:07.833716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:07.931784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd89a4d-ced8-42bc-a84b-038e47c6a36c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:07.938280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49b2c-d35d-4725-8e4f-01c0842dfef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:07.938786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49b2c-d35d-4725-8e4f-01c0842dfef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.939118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49b2c-d35d-4725-8e4f-01c0842dfef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.939434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49b2c-d35d-4725-8e4f-01c0842dfef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.940323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f49b2c-d35d-4725-8e4f-01c0842dfef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:07.942137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f49b2c-d35d-4725-8e4f-01c0842dfef9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:07.956872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 919/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:07.958835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:07.959120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:07.959646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:07.980209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 926/1845] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:07.981783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:07.983463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:07.998467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c69d86-ac10-445e-83ea-81c415dbfd74 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.004096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.005001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.005114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.005343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] PATH_INFO: `/v3/projects/50862e13563d4a6a9ca6ee91a09c2423` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.006939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50862e13563d4a6a9ca6ee91a09c2423)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.009006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.012416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] MFA Rules not processed for user `46823c0ddbe14f1792be2fab7b931306`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:08.015065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.064688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50862e13563d4a6a9ca6ee91a09c2423', 'request_id': 'req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.072251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50862e13563d4a6a9ca6ee91a09c2423', 'request_id': 'req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.083940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50862e13563d4a6a9ca6ee91a09c2423', 'request_id': 'req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.097771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] The token cache is being invalidate because project 50862e13563d4a6a9ca6ee91a09c2423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:08.098218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.099524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.105949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 927/1847] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:07 2022] DELETE /identity/v3/projects/50862e13563d4a6a9ca6ee91a09c2423 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:08.106120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 920/1847] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:08.106203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.106275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.106344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.131339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfe8a9-f446-4268-95c3-7df51238c09a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.162167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:08.186039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7747b6-f2a9-40e0-b4d6-f6213c3f3ef4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.192233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.192911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.193245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.193985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] PATH_INFO: `/v3/projects/714ff544a3db484abf71c9f97f2bc0e2/users/ab638d0d58134112a96eb72a9904eadb/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.195929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=714ff544a3db484abf71c9f97f2bc0e2, user_id=ab638d0d58134112a96eb72a9904eadb, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.198842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.213640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 928/1848] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:08 2022] PUT /identity/v3/projects/714ff544a3db484abf71c9f97f2bc0e2/users/ab638d0d58134112a96eb72a9904eadb/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:08.214788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.215762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:08.218658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.223428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864aed41-d72d-4f20-8983-81fd1d63e284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.224827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aed41-d72d-4f20-8983-81fd1d63e284 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.225270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aed41-d72d-4f20-8983-81fd1d63e284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.225742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aed41-d72d-4f20-8983-81fd1d63e284 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.227134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864aed41-d72d-4f20-8983-81fd1d63e284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.229518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864aed41-d72d-4f20-8983-81fd1d63e284 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.271187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 921/1849] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:08 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:08.272655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111443c-224b-43c5-b155-822558154d86 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.273127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111443c-224b-43c5-b155-822558154d86 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.273478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111443c-224b-43c5-b155-822558154d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.311796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7111443c-224b-43c5-b155-822558154d86 None None] MFA Rules not processed for user `ab638d0d58134112a96eb72a9904eadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:08.314585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7111443c-224b-43c5-b155-822558154d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.327509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afd1c-14a6-40a4-a952-65fa3a5679ad admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.384690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.385433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.385716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.385983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] PATH_INFO: `/v3/users/10ddbe2b00e640d8aadc85d085618214` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.387376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ddbe2b00e640d8aadc85d085618214)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.389356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.415085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 922/1850] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:08.426239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111443c-224b-43c5-b155-822558154d86 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.427754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7111443c-224b-43c5-b155-822558154d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.441767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111443c-224b-43c5-b155-822558154d86 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.446425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.446986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.447312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.447610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.447813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ddbe2b00e640d8aadc85d085618214'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.448270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.450157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.460253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ddbe2b00e640d8aadc85d085618214'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.460923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ddbe2b00e640d8aadc85d085618214'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.461707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ddbe2b00e640d8aadc85d085618214'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.474386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ddbe2b00e640d8aadc85d085618214'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:08.481915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 923/1851] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:08 2022] POST /identity/v3/users => generated 297 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:08.483670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.485504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.500342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 929/1852] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:08 2022] DELETE /identity/v3/users/10ddbe2b00e640d8aadc85d085618214 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:08.502027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.503545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.510999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:08.556290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a0de-9346-4e79-b94c-5db7e3265788 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.597450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549ec6d-7b92-4d48-ad5c-efe470ba3cde admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.602831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.603557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.603902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.604241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.605318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.607025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.615603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.616365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.616747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.617115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.618974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.619844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 930/1853] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:08.623832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.634725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b5027-cd5f-49d6-ba4b-f01d5cf2e241 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.640607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.641472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.642246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.642620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] PATH_INFO: `/v3/projects/f9d5f85bb23f4437a7fb7c339f0a1369/users/a0cecadd26204b618c2f9e4faccc3929/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.660237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:08.678588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d5f85bb23f4437a7fb7c339f0a1369, user_id=a0cecadd26204b618c2f9e4faccc3929, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.681611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.705936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 931/1854] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:08 2022] PUT /identity/v3/projects/f9d5f85bb23f4437a7fb7c339f0a1369/users/a0cecadd26204b618c2f9e4faccc3929/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:08.706922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.726661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.728159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.739066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 924/1855] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:08.838839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e68078-f9c0-468e-92ab-045e4c7a1b2a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.846779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.847728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.847934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.848206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.848924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.850594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.853523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.863912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 932/1856] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:08.867382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7466a5f-63d6-4e3f-b302-3dd719e94203 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.873028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.898689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53abc0-285c-4d70-8b43-adadc17e8ddf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:08.908538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.909216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.909491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.909759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] PATH_INFO: `/v3/projects/f9d5f85bb23f4437a7fb7c339f0a1369/users/a0cecadd26204b618c2f9e4faccc3929/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.911827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d5f85bb23f4437a7fb7c339f0a1369, user_id=a0cecadd26204b618c2f9e4faccc3929, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:08.914460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:08.931922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 933/1857] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:08 2022] PUT /identity/v3/projects/f9d5f85bb23f4437a7fb7c339f0a1369/users/a0cecadd26204b618c2f9e4faccc3929/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:08.931922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:08.931922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:08.940359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:08.941284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:08.941559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:08.941825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:08.944256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.011657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80cf7a-0c52-46b4-ac7e-6e8d3a78f410 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:09.017377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:09.018009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.018287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.018556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] PATH_INFO: `/v3/projects/7b4f3ae4d36b46b5bb21e2eadfdc396f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.019829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4f3ae4d36b46b5bb21e2eadfdc396f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:09.021782 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:09.024412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:09.029017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:09.030923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa173f-07f4-4545-aee7-fe7285c0238d service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:09.043527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 925/1858] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:09.046518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73a5b8-eb1e-4535-91ef-1a88fe001649 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.046969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73a5b8-eb1e-4535-91ef-1a88fe001649 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.047395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73a5b8-eb1e-4535-91ef-1a88fe001649 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.049627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 926/1859] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:45:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:45:09.050843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.051546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.051546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.084462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] MFA Rules not processed for user `a0cecadd26204b618c2f9e4faccc3929`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:09.085940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.129291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4f3ae4d36b46b5bb21e2eadfdc396f', 'request_id': 'req-40554678-3bd1-44b8-a81a-966634cebfe0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.147179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4f3ae4d36b46b5bb21e2eadfdc396f', 'request_id': 'req-40554678-3bd1-44b8-a81a-966634cebfe0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.159110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4f3ae4d36b46b5bb21e2eadfdc396f', 'request_id': 'req-40554678-3bd1-44b8-a81a-966634cebfe0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.173556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:09.175002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] The token cache is being invalidate because project 7b4f3ae4d36b46b5bb21e2eadfdc396f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:09.175299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.176360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-40554678-3bd1-44b8-a81a-966634cebfe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.178489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 934/1860] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:08 2022] DELETE /identity/v3/projects/7b4f3ae4d36b46b5bb21e2eadfdc396f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:09.180109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.180395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.180664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.186666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 927/1861] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:09.187762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:09.206040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2cc5b-c19c-406e-bcc0-ad771fb4de29 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:09.208717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:09.210109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.278396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:09.279314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.280070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.280070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] PATH_INFO: `/v3/users/30406093669446d98af75f3dee7bb12b` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.281593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=30406093669446d98af75f3dee7bb12b)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:09.283922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:09.311550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 935/1862] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:45:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:09.312623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:09.315554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.349149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30406093669446d98af75f3dee7bb12b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.365208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30406093669446d98af75f3dee7bb12b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.366472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30406093669446d98af75f3dee7bb12b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.378848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30406093669446d98af75f3dee7bb12b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.381270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30406093669446d98af75f3dee7bb12b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:09.414055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 928/1863] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:09 2022] DELETE /identity/v3/users/30406093669446d98af75f3dee7bb12b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:09.415239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:09.418282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.464102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15681bb-b0fe-4d7b-b943-e4a8699892e7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:09.542146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:09.566275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:09.567010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.569444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.570041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.573905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.614539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be238-3fac-4c17-b27f-c12c6da42ebc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:09.621341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:09.622588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.622885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.623271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.627000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.643012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:09.651970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:09.666495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 929/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:09.691394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:09.692960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:09.708012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 936/1865] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:09.802471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:09.804091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.811793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:09.816768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bae155-a6f7-479a-921c-aeb0085381a1 service placement] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:09.821377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:09.822096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:09.822755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:09.823201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:09.826050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:09.944207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:09.946393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:09.959349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 930/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:10.013076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.015043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.039032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.041373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.119665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a29515-51f3-40cc-babe-812cd7104917 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.128439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.129093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.129372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.129903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] PATH_INFO: `/v3/users/dd777be73a7d446ab40a3eb730ccdb49` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.131042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd777be73a7d446ab40a3eb730ccdb49)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.133245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.133454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9c14-d282-4b7b-a5aa-aff000ddcb9d service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.139069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.140006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.140450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.140847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] PATH_INFO: `/v3/projects/8e58edc84dd1478fb8c339bca269229a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.144021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e58edc84dd1478fb8c339bca269229a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.146484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.207276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e58edc84dd1478fb8c339bca269229a', 'request_id': 'req-c9ef5111-9a74-4f48-aad4-30757c5d24c4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.214231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e58edc84dd1478fb8c339bca269229a', 'request_id': 'req-c9ef5111-9a74-4f48-aad4-30757c5d24c4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.217967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd777be73a7d446ab40a3eb730ccdb49'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.225011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e58edc84dd1478fb8c339bca269229a', 'request_id': 'req-c9ef5111-9a74-4f48-aad4-30757c5d24c4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.238997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd777be73a7d446ab40a3eb730ccdb49'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.238997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd777be73a7d446ab40a3eb730ccdb49'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.239639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd777be73a7d446ab40a3eb730ccdb49'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.240815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] The token cache is being invalidate because project 8e58edc84dd1478fb8c339bca269229a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:10.241272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.242038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c9ef5111-9a74-4f48-aad4-30757c5d24c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.248080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 931/1867] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:10 2022] DELETE /identity/v3/projects/8e58edc84dd1478fb8c339bca269229a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:10.248535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd777be73a7d446ab40a3eb730ccdb49'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.249377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35967223-8a4a-42a8-a571-624df3a632d1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.249660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35967223-8a4a-42a8-a571-624df3a632d1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.249935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35967223-8a4a-42a8-a571-624df3a632d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.272251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-35967223-8a4a-42a8-a571-624df3a632d1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:10.273730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-35967223-8a4a-42a8-a571-624df3a632d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.274511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 937/1868] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:10 2022] DELETE /identity/v3/users/dd777be73a7d446ab40a3eb730ccdb49 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:10.295003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.296589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.313566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:10.323794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:10.377903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 932/1869] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:10.379780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.380073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.380349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.394157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59eb09-b96b-43f7-bd3d-e5a867038ce2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.406750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.408214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.408530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.408918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] PATH_INFO: `/v3/users/b9fe5383178d4164a71e258eba1c06ba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.410323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fe5383178d4164a71e258eba1c06ba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.412814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.412966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:10.414239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.423136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 933/1870] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:10.427328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.436669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb773ef-d438-4ff6-9ac7-d5022f8ed65d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.442011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.442616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.442893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.443162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.443894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:10.444145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.445898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.458866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 934/1871] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:45:10.463707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.465648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fe5383178d4164a71e258eba1c06ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.475920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fe5383178d4164a71e258eba1c06ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.476769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fe5383178d4164a71e258eba1c06ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.477281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726002c-ea91-4fba-a46c-1299abbbf60e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.477583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fe5383178d4164a71e258eba1c06ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.484192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d87d32-3fa4-41ee-bec2-ccf0d1185496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.484834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d87d32-3fa4-41ee-bec2-ccf0d1185496 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.485138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d87d32-3fa4-41ee-bec2-ccf0d1185496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.485523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d87d32-3fa4-41ee-bec2-ccf0d1185496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.486380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d87d32-3fa4-41ee-bec2-ccf0d1185496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.488745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fe5383178d4164a71e258eba1c06ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.488898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d87d32-3fa4-41ee-bec2-ccf0d1185496 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.527215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 938/1872] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:10 2022] DELETE /identity/v3/users/b9fe5383178d4164a71e258eba1c06ba => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:10.531860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.533257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.540993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 935/1873] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:10 2022] POST /identity/v3/projects => generated 409 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:10.542484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.542909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.543239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.565620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:10.567300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.607398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf24f0a-63bb-4146-92e4-8d95249a5ad1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.640974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 936/1874] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:10.640974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.640974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.647998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:10.658607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb124822-28ca-4b0b-8bb2-91549eb9750b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.672493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.674471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.675992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.676371 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.677125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.680689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.695950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.696875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.697761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.698317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] PATH_INFO: `/v3/projects/417bf94ed53447b9927e8a20535464ab` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.699667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=417bf94ed53447b9927e8a20535464ab)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.701780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.712896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 937/1875] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:10 2022] POST /identity/v3/users => generated 305 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:10.712896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.713976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.762899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417bf94ed53447b9927e8a20535464ab', 'request_id': 'req-7792121c-b769-4c46-8761-f3d43078c3ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.769304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417bf94ed53447b9927e8a20535464ab', 'request_id': 'req-7792121c-b769-4c46-8761-f3d43078c3ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.778440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05fd05-bc54-4df8-8867-e7550c6ee39f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.779584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417bf94ed53447b9927e8a20535464ab', 'request_id': 'req-7792121c-b769-4c46-8761-f3d43078c3ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.799726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] The token cache is being invalidate because project 417bf94ed53447b9927e8a20535464ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:10.800019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.800689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:10.802363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 939/1876] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:10 2022] DELETE /identity/v3/projects/417bf94ed53447b9927e8a20535464ab => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:10.803120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.804487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.843832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:10.889712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.890557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.891168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.891795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.895204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.912417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792121c-b769-4c46-8761-f3d43078c3ca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.926030 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.927471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.928293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.929086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.929503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.929942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.930743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.932833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.938074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 938/1877] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:10.939749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.941233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:10.951369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 940/1878] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:10.956236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:10.957060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96ad49-4f0c-4028-9d7d-2a6e736ce30d service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.962723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.963359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.963669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.963940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] PATH_INFO: `/v3/users/46823c0ddbe14f1792be2fab7b931306` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.965127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46823c0ddbe14f1792be2fab7b931306)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.967101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.968985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff44e2e-db1f-4a5e-a358-f7d28abf659a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:10.975622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:10.976369 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:10.976722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:10.977441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:10.978323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:10.980021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:10.996187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 941/1879] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:10.997819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.011145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade3b21-b841-42aa-908e-036505848f79 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.016826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a0944-27fd-4748-99ab-4cc6295ae6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.018171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a0944-27fd-4748-99ab-4cc6295ae6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.018384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a0944-27fd-4748-99ab-4cc6295ae6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.018648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a0944-27fd-4748-99ab-4cc6295ae6f2 admin admin] PATH_INFO: `/v3/projects/fa4391c518004195b3f62c6d622f8f28/users/47a8e8c2c44b4908bd7c4b00a9ff1487/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.053221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a0944-27fd-4748-99ab-4cc6295ae6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4391c518004195b3f62c6d622f8f28, user_id=47a8e8c2c44b4908bd7c4b00a9ff1487, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.056965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a0944-27fd-4748-99ab-4cc6295ae6f2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.062764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46823c0ddbe14f1792be2fab7b931306'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.074824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46823c0ddbe14f1792be2fab7b931306'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.075892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46823c0ddbe14f1792be2fab7b931306'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.091709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 942/1880] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:10 2022] PUT /identity/v3/projects/fa4391c518004195b3f62c6d622f8f28/users/47a8e8c2c44b4908bd7c4b00a9ff1487/roles/87f1831257824560813f954348534540 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:11.093560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.093893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.094087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.103095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46823c0ddbe14f1792be2fab7b931306'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.105134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46823c0ddbe14f1792be2fab7b931306'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.130290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] MFA Rules not processed for user `47a8e8c2c44b4908bd7c4b00a9ff1487`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:11.131989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.140536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 939/1881] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:10 2022] DELETE /identity/v3/users/46823c0ddbe14f1792be2fab7b931306 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:11.152636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:11.212400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 943/1882] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:11.216753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.218181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.288455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ffc86-3b5f-4e16-a9eb-e1182bd819f5 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.293322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.293942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.294327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.294502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.295283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.298388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.330400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 940/1883] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:11 2022] POST /identity/v3/projects => generated 407 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:11.335501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.411260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253ff5d-97fb-4c34-a2f2-52a809133775 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.449749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.452449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.503035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.503900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.504350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.504780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.505569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.508008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.523027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:11.539683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 944/1884] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:11 2022] POST /identity/v3/users => generated 310 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:11.540580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.544657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.557805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88860bb8-8ee6-45d3-9204-176c7eb613dc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.563045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.563627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.563933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.564208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] PATH_INFO: `/v3/projects/1ede4436931542f4a61b528bb8a939c9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.565492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ede4436931542f4a61b528bb8a939c9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.566357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00573c-6a43-46c6-842b-2f22705c7e48 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.567350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.571382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.572650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.573039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.573526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] PATH_INFO: `/v3/projects/7eec17aa13734b8a9eca85b9b8161a5d` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.575600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eec17aa13734b8a9eca85b9b8161a5d)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.577507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.627733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ede4436931542f4a61b528bb8a939c9', 'request_id': 'req-0005cc34-9f8a-40b7-91bd-a9884d4006db'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.629471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:11.632067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eec17aa13734b8a9eca85b9b8161a5d', 'request_id': 'req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.635329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ede4436931542f4a61b528bb8a939c9', 'request_id': 'req-0005cc34-9f8a-40b7-91bd-a9884d4006db'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.637526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eec17aa13734b8a9eca85b9b8161a5d', 'request_id': 'req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.646344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ede4436931542f4a61b528bb8a939c9', 'request_id': 'req-0005cc34-9f8a-40b7-91bd-a9884d4006db'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.652713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eec17aa13734b8a9eca85b9b8161a5d', 'request_id': 'req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.662972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] The token cache is being invalidate because project 1ede4436931542f4a61b528bb8a939c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:11.663257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.663939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.664949 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] The token cache is being invalidate because project 7eec17aa13734b8a9eca85b9b8161a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:11.665236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.665429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 945/1885] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:11 2022] DELETE /identity/v3/projects/1ede4436931542f4a61b528bb8a939c9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:11.666185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.666288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1f7ec398-ac20-4f19-b89e-40f53a9bf3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.667927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 941/1886] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:11 2022] DELETE /identity/v3/projects/7eec17aa13734b8a9eca85b9b8161a5d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:11.668904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.672314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.672593 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.672908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.692643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:11.693867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.755688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005cc34-9f8a-40b7-91bd-a9884d4006db admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.787143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 942/1887] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:11.788114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.792360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.810866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19194006-7dc4-484f-9c43-a3b941e5477d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.830472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.831474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.831860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.832637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] PATH_INFO: `/v3/users/a0cecadd26204b618c2f9e4faccc3929` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.834273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0cecadd26204b618c2f9e4faccc3929)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.837076 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.838233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a82449-caa4-4b0b-b1c4-b6176a9baf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.838974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a82449-caa4-4b0b-b1c4-b6176a9baf17 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.839253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a82449-caa4-4b0b-b1c4-b6176a9baf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.840212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a82449-caa4-4b0b-b1c4-b6176a9baf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.840212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a82449-caa4-4b0b-b1c4-b6176a9baf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.841777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a82449-caa4-4b0b-b1c4-b6176a9baf17 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.862648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 946/1888] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:11.864594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.865073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.865412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.891143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:11.893029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.903064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0cecadd26204b618c2f9e4faccc3929'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.905458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 947/1889] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:11.906332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.908390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.920945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0cecadd26204b618c2f9e4faccc3929'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.922105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0cecadd26204b618c2f9e4faccc3929'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.925347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:11.933765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0cecadd26204b618c2f9e4faccc3929'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.935603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0cecadd26204b618c2f9e4faccc3929'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:11.935857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479439e-96f6-43a2-881d-73f3d29c691c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:11.953625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:11.954888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:11.955173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:11.955443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] PATH_INFO: `/v3/projects/fdbfd7c131134438a6dd37d6c6407113/users/9299a804af8a40fd9cad8a304f50f660/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:11.957702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 943/1890] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:11 2022] DELETE /identity/v3/users/a0cecadd26204b618c2f9e4faccc3929 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:11.961883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:11.963355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:11.973893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbfd7c131134438a6dd37d6c6407113, user_id=9299a804af8a40fd9cad8a304f50f660, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:11.976820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:11.995670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 948/1891] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:11 2022] PUT /identity/v3/projects/fdbfd7c131134438a6dd37d6c6407113/users/9299a804af8a40fd9cad8a304f50f660/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:12.000756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.004248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.056687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608690-279d-494d-a77c-80783f7b932c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.069454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:12.085412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583abb-adc8-4321-9dc2-be237870dee1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.092115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.092917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.093368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.093860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.094596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.096794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.113768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 949/1892] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:12.118802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.143466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb9a7-fbf4-4c6d-988b-286c199a30a7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.152626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be64d4-bda9-4a78-b1c2-f81efc4eb837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.153616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be64d4-bda9-4a78-b1c2-f81efc4eb837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.154057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be64d4-bda9-4a78-b1c2-f81efc4eb837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.154525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be64d4-bda9-4a78-b1c2-f81efc4eb837 admin admin] PATH_INFO: `/v3/projects/fdbfd7c131134438a6dd37d6c6407113/users/9299a804af8a40fd9cad8a304f50f660/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.157318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be64d4-bda9-4a78-b1c2-f81efc4eb837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbfd7c131134438a6dd37d6c6407113, user_id=9299a804af8a40fd9cad8a304f50f660, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.160906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be64d4-bda9-4a78-b1c2-f81efc4eb837 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.162143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.162989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.163256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.163737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.164489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:12.164885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.166564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.180245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 950/1893] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:12 2022] PUT /identity/v3/projects/fdbfd7c131134438a6dd37d6c6407113/users/9299a804af8a40fd9cad8a304f50f660/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:12.192067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.195024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.196074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.200060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 944/1894] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:12.205277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.219462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45cb7-1d3d-4452-bd36-a66a0c1569ea admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.250743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] MFA Rules not processed for user `9299a804af8a40fd9cad8a304f50f660`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:12.252461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.316957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.317822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.318299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.318706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.320122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.322341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.346623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:12.354953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 945/1895] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:12.358767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 951/1896] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:12.361347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.372717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.374318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.436006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af7e6d-0cee-46a3-a62b-d86a73fb3f32 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.436220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9591419-65fc-4201-aa80-cd748fe33ffb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.441187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.441787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.442084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.442416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.442966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.444911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.472216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 946/1897] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:12.472216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.472763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.484663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533ae6-9e92-487c-aba9-51fa08c2eb43 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.490121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.490811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.491088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.491381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e23547-eb29-4312-9c0c-1ddbdeb1a42c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.491526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] PATH_INFO: `/v3/projects/f9d5f85bb23f4437a7fb7c339f0a1369` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.492059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e23547-eb29-4312-9c0c-1ddbdeb1a42c service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.492333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e23547-eb29-4312-9c0c-1ddbdeb1a42c service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.492579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d5f85bb23f4437a7fb7c339f0a1369)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.492677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e23547-eb29-4312-9c0c-1ddbdeb1a42c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.494545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.524384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:12.543131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d5f85bb23f4437a7fb7c339f0a1369', 'request_id': 'req-f63784db-db92-44e8-b641-509e21e74199'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:12.549085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d5f85bb23f4437a7fb7c339f0a1369', 'request_id': 'req-f63784db-db92-44e8-b641-509e21e74199'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:12.556382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d5f85bb23f4437a7fb7c339f0a1369', 'request_id': 'req-f63784db-db92-44e8-b641-509e21e74199'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:12.559720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e23547-eb29-4312-9c0c-1ddbdeb1a42c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.561305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e23547-eb29-4312-9c0c-1ddbdeb1a42c service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.568429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] The token cache is being invalidate because project f9d5f85bb23f4437a7fb7c339f0a1369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:12.568711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:12.569350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:12.571032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 947/1898] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:12 2022] DELETE /identity/v3/projects/f9d5f85bb23f4437a7fb7c339f0a1369 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:12.571622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.571733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 952/1899] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:12 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:12.572969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.575842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f337e7a-f1d7-4fae-94c2-00b6fdc27361 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.576126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f337e7a-f1d7-4fae-94c2-00b6fdc27361 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.576398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f337e7a-f1d7-4fae-94c2-00b6fdc27361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.596998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2f337e7a-f1d7-4fae-94c2-00b6fdc27361 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:12.598189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2f337e7a-f1d7-4fae-94c2-00b6fdc27361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.641129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63784db-db92-44e8-b641-509e21e74199 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.649190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.650131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.651704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.652170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.653013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.657939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.678010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 948/1900] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:12.680500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.690761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 953/1901] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:12.693418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0f9b-6cd9-4afc-b532-810c2dc32fd0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.699157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.699797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.700075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.700517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.700879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.701168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.701527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.702024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.702887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.705365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:12.717483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 949/1902] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:12.723523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.728512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:12.730081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.736293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfa787-bd0e-48ce-be38-c4835377f1d3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.743151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 954/1903] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:12.746668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.746668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.746895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.747050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] PATH_INFO: `/v3/projects/6297864e0dae464b82a352fb4778e2c9/users/820ed2ccec6842dba7fb07eea204ffec/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.749446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.761665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3216705-67b2-4678-a05e-f3096741a969 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.765254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6297864e0dae464b82a352fb4778e2c9, user_id=820ed2ccec6842dba7fb07eea204ffec, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.767879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.776109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.777604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.778123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.780854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:12.781170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.781352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.784083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.810965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 955/1904] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:12.810965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.811343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 950/1905] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:12 2022] PUT /identity/v3/projects/6297864e0dae464b82a352fb4778e2c9/users/820ed2ccec6842dba7fb07eea204ffec/roles/87f1831257824560813f954348534540 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:12.812946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:12.817455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.883154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d060-83ea-470e-8b8c-7d3444cb72f3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.885795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2be96-c3c3-4491-87c6-89287018dcae admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:12.887819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc5fe3-4310-48e9-a1ac-b3cdfaa473b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.888419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc5fe3-4310-48e9-a1ac-b3cdfaa473b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.888699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc5fe3-4310-48e9-a1ac-b3cdfaa473b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.888970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc5fe3-4310-48e9-a1ac-b3cdfaa473b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.889607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc5fe3-4310-48e9-a1ac-b3cdfaa473b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:12.891566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc5fe3-4310-48e9-a1ac-b3cdfaa473b9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:12.918874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 951/1906] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:12.920074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.920364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.920694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.935350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:12.936091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:12.936371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:12.936637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:12.938391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.949060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] MFA Rules not processed for user `820ed2ccec6842dba7fb07eea204ffec`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:12.950544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:12.952518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:13.003146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.005108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.014294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 956/1907] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:12 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:13.015092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.028087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477512e6-8482-4a52-8b56-a3792eb8d014 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.046103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 952/1908] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:13.049533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.095627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.096398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.096679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.096947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.097656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.099870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.119393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:13.136203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 957/1909] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:13 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:13.141122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.143058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678d59c-d742-434a-82d3-e47e341a38e6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.149061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.149680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.150503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.150781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.151740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.153789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.153912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7af62-7356-47cb-b11f-cbf24442facf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.161837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.162515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.162781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.163051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.163785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.165423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.178130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 958/1910] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:13.182411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.195169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 953/1911] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:13 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:13.195337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2f21c-6312-4dab-9286-77a746b5fcb5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.196476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.268865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.270254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.270540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.270809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.271460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.273701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.287527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 959/1912] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:13.290618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.303228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0e2a4-aa21-445f-8a6b-af678a8b6a2c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.308343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82970831-8bb5-48ac-880a-eb582ecef7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.309009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82970831-8bb5-48ac-880a-eb582ecef7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.309329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82970831-8bb5-48ac-880a-eb582ecef7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.309602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82970831-8bb5-48ac-880a-eb582ecef7d2 admin admin] PATH_INFO: `/v3/projects/06a0252a4fba4f1ebfa9f25a2c6e39a4/users/5f294ab83780460f9679c7655c97a6e4/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.311257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740ceee-b47a-4213-ac57-436818aa2f0a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.319044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.322565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.322565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.322565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.322565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.328376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82970831-8bb5-48ac-880a-eb582ecef7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a0252a4fba4f1ebfa9f25a2c6e39a4, user_id=5f294ab83780460f9679c7655c97a6e4, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.332558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82970831-8bb5-48ac-880a-eb582ecef7d2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.337457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.356680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 960/1913] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:13 2022] PUT /identity/v3/projects/06a0252a4fba4f1ebfa9f25a2c6e39a4/users/5f294ab83780460f9679c7655c97a6e4/roles/87f1831257824560813f954348534540 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:13.369516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.370086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.370457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.381698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 954/1914] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:13 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:13.385903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.388295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:13.397543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:13.400829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:13.408130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b2907-28a5-435f-98d9-cdec98671cd7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.504995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 961/1915] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:13.524899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.526411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:13.552675 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.553306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.553587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.553859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.554702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.556932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.572829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 955/1916] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:13.576601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.589143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d1a26-5e0b-4b5e-9245-cbea1b488b23 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.596141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.596814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.597093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.597368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] PATH_INFO: `/v3/projects/b027a9bd29ff436293420dc0ebaca85e/users/404979ce8c4b4e6899c4983fe93ae5dd/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.615518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eda891-67a4-45d2-b1ee-a98be090f99b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.620991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.621547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b027a9bd29ff436293420dc0ebaca85e, user_id=404979ce8c4b4e6899c4983fe93ae5dd, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.621668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.621893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.622169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.624259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.633044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.634523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.637634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 956/1917] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:13 2022] PUT /identity/v3/projects/b027a9bd29ff436293420dc0ebaca85e/users/404979ce8c4b4e6899c4983fe93ae5dd/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:13.641536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.645046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 962/1918] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:13.684235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:13.709563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce7cf-d4bd-41d2-afdc-bf56ec0c82ac admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.714469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727acdd-fe70-46c6-bc67-f7aa34cc72bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.715070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727acdd-fe70-46c6-bc67-f7aa34cc72bc admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.715344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727acdd-fe70-46c6-bc67-f7aa34cc72bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.715646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727acdd-fe70-46c6-bc67-f7aa34cc72bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.716282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727acdd-fe70-46c6-bc67-f7aa34cc72bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.717855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727acdd-fe70-46c6-bc67-f7aa34cc72bc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.729144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 957/1919] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:13.733790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.744998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210938e0-2bc8-46e6-8fb4-8aba8322c3ae service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.749788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.750437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.750721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.750991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] PATH_INFO: `/v3/projects/b027a9bd29ff436293420dc0ebaca85e/users/404979ce8c4b4e6899c4983fe93ae5dd/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.752983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b027a9bd29ff436293420dc0ebaca85e, user_id=404979ce8c4b4e6899c4983fe93ae5dd, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.755953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.766703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 963/1920] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:13 2022] PUT /identity/v3/projects/b027a9bd29ff436293420dc0ebaca85e/users/404979ce8c4b4e6899c4983fe93ae5dd/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:13.777981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.778348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.778552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.809010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] MFA Rules not processed for user `404979ce8c4b4e6899c4983fe93ae5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:13.810241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:13.890545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 958/1921] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:13.902609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:13.904876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:13.966597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2179-c6ca-4f2d-ba9a-6413067e6f98 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:13.970856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166b22-9730-4ebd-b666-125f099e46b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:13.971462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166b22-9730-4ebd-b666-125f099e46b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:13.971769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166b22-9730-4ebd-b666-125f099e46b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:13.972768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166b22-9730-4ebd-b666-125f099e46b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:13.984014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166b22-9730-4ebd-b666-125f099e46b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:13.984734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166b22-9730-4ebd-b666-125f099e46b6 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:13.997133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 964/1922] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:14.102038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:14.113551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03004a1c-d565-4dbf-b18f-9e7db1235b30 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:14.117328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:14.118016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:14.118364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:14.118706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:14.179568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:14.181469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:14.192821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 959/1923] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:14.464891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:14.465379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:14.465505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:14.494421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] MFA Rules not processed for user `5f294ab83780460f9679c7655c97a6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:14.495571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:14.526646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:14.579862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 965/1924] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:14.587336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:14.596640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139227e-5e32-4d37-bb5e-951f65171fe9 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:14.600312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a39cf-94f1-46e0-877e-571f91232baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:14.600974 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a39cf-94f1-46e0-877e-571f91232baa service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:14.601273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a39cf-94f1-46e0-877e-571f91232baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:14.601526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a39cf-94f1-46e0-877e-571f91232baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:14.609984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a39cf-94f1-46e0-877e-571f91232baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:14.612076 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a39cf-94f1-46e0-877e-571f91232baa service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:14.622794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 960/1925] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:18.007236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:18.080871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0be7cc-18eb-4258-ab14-8e2fac13fd22 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:18.092435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:18.094023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:18.095210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:18.096320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] PATH_INFO: `/v3/users/820ed2ccec6842dba7fb07eea204ffec` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:18.111721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820ed2ccec6842dba7fb07eea204ffec)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:18.111721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:18.126500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bb061-1bfe-47b4-a07e-92fa6b6390a4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:18.127031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bb061-1bfe-47b4-a07e-92fa6b6390a4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:18.127031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bb061-1bfe-47b4-a07e-92fa6b6390a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:18.147786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5b3bb061-1bfe-47b4-a07e-92fa6b6390a4 None None] MFA Rules not processed for user `5f294ab83780460f9679c7655c97a6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:18.149117 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5b3bb061-1bfe-47b4-a07e-92fa6b6390a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:18.158722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 961/1926] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:18.171738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820ed2ccec6842dba7fb07eea204ffec'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:18.182498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820ed2ccec6842dba7fb07eea204ffec'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:18.183230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820ed2ccec6842dba7fb07eea204ffec'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:18.183912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820ed2ccec6842dba7fb07eea204ffec'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:18.193066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820ed2ccec6842dba7fb07eea204ffec'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:18.213522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 966/1927] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:18 2022] DELETE /identity/v3/users/820ed2ccec6842dba7fb07eea204ffec => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:18.218047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:18.218406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:18.218683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:18.242062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:18.244050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:18.280966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:18.319208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 962/1928] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:18.336153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:18.337602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:18.400520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:18.404732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f252a5-90c3-4c85-b479-e89a0f578452 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:18.411045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:18.411789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:18.412604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:18.412878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:18.422837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:18.424311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:18.434015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 967/1929] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:18.937840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:18.951083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:18.968380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca668-978b-47bd-b9bc-f510b2177610 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:18.975421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:18.976414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:18.976875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:18.977324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] PATH_INFO: `/v3/projects/6297864e0dae464b82a352fb4778e2c9` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:18.979310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6297864e0dae464b82a352fb4778e2c9)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:18.981634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:19.073088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6297864e0dae464b82a352fb4778e2c9', 'request_id': 'req-93391268-16c1-4da4-9e62-f61babac0556'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.112164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6297864e0dae464b82a352fb4778e2c9', 'request_id': 'req-93391268-16c1-4da4-9e62-f61babac0556'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.127861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6297864e0dae464b82a352fb4778e2c9', 'request_id': 'req-93391268-16c1-4da4-9e62-f61babac0556'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.150417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] The token cache is being invalidate because project 6297864e0dae464b82a352fb4778e2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:19.150782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.152850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.153313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:19.154996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.158708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 963/1930] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:18 2022] DELETE /identity/v3/projects/6297864e0dae464b82a352fb4778e2c9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:19.161150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:19.163251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.170542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8af5-0e2f-443f-87a7-d36b3f4dfef5 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:19.255377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:19.256493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.257036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.257437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] PATH_INFO: `/v3/users/05c6b9c83002419198d4fc106cd5c509` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:19.258908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c6b9c83002419198d4fc106cd5c509)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:19.261543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:19.268595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391268-16c1-4da4-9e62-f61babac0556 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:19.274293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:19.274937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.275211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.275496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] PATH_INFO: `/v3/users/404979ce8c4b4e6899c4983fe93ae5dd` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:19.276668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404979ce8c4b4e6899c4983fe93ae5dd)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:19.279345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:19.327207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c6b9c83002419198d4fc106cd5c509'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.335537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c6b9c83002419198d4fc106cd5c509'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.336318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c6b9c83002419198d4fc106cd5c509'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.336983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c6b9c83002419198d4fc106cd5c509'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.348078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5b3e16dd-c49c-4b01-a4dd-cfe3ed813979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c6b9c83002419198d4fc106cd5c509'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.348229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404979ce8c4b4e6899c4983fe93ae5dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.359433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404979ce8c4b4e6899c4983fe93ae5dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.360344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404979ce8c4b4e6899c4983fe93ae5dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.368178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:19.371018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404979ce8c4b4e6899c4983fe93ae5dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.371883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404979ce8c4b4e6899c4983fe93ae5dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.386645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 968/1931] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:19 2022] DELETE /identity/v3/users/05c6b9c83002419198d4fc106cd5c509 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:19.391803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.392086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.392409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:19.400853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 964/1932] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:19 2022] DELETE /identity/v3/users/404979ce8c4b4e6899c4983fe93ae5dd => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:19.429518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:19.430943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.521767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 969/1933] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:19.553080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:19.554523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.615667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33761-c18b-4c23-a070-e162a68173c2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:19.619669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28fde0-aa63-4952-b818-870c1da38225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:19.620281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28fde0-aa63-4952-b818-870c1da38225 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.620587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28fde0-aa63-4952-b818-870c1da38225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.620860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28fde0-aa63-4952-b818-870c1da38225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:19.630431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28fde0-aa63-4952-b818-870c1da38225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:19.632259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28fde0-aa63-4952-b818-870c1da38225 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:19.642114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 965/1934] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:19.706112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:19.708239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.729300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aec944-b102-4117-9ef7-d204b3bfc0d0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:19.740258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:19.741451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.741809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.742008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] PATH_INFO: `/v3/projects/b027a9bd29ff436293420dc0ebaca85e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:19.743204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] RBAC: Authorizing `identity:delete_project(project_id=b027a9bd29ff436293420dc0ebaca85e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:19.745179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:19.752776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:19.799822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b027a9bd29ff436293420dc0ebaca85e', 'request_id': 'req-d8fe6f64-40f4-4323-856f-ad9e154468db'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.805984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b027a9bd29ff436293420dc0ebaca85e', 'request_id': 'req-d8fe6f64-40f4-4323-856f-ad9e154468db'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.814763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b027a9bd29ff436293420dc0ebaca85e', 'request_id': 'req-d8fe6f64-40f4-4323-856f-ad9e154468db'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.827959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] The token cache is being invalidate because project b027a9bd29ff436293420dc0ebaca85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:19.828331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.828919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:19.830423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 970/1935] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:19 2022] DELETE /identity/v3/projects/b027a9bd29ff436293420dc0ebaca85e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:19.835636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f1ae2-b6cd-447a-ba81-9042038c3e8a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.836020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f1ae2-b6cd-447a-ba81-9042038c3e8a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.836367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f1ae2-b6cd-447a-ba81-9042038c3e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:19.856874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-6e4f1ae2-b6cd-447a-ba81-9042038c3e8a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:19.858184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6e4f1ae2-b6cd-447a-ba81-9042038c3e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.948278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 966/1936] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:19.958175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:19.960091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:19.962357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:19.962825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:19.963173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.000761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:20.002872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.003338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe6f64-40f4-4323-856f-ad9e154468db admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.019685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 967/1937] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:20.019861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.019861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.019861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.020118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] PATH_INFO: `/v3/projects/7ed32bd8cf964c67be3d5b02dcbd86c2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.021346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed32bd8cf964c67be3d5b02dcbd86c2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.023963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.024944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.047539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9480a1-04dc-406c-a80f-9d6723dd84e5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.052646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.053041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.053312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.053579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.054241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:20.054540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.056302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.070732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 968/1938] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:20.084659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.095407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed32bd8cf964c67be3d5b02dcbd86c2', 'request_id': 'req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.096644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873388d4-48b4-4a8f-b7eb-7bdee6976218 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.103065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed32bd8cf964c67be3d5b02dcbd86c2', 'request_id': 'req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.117064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed32bd8cf964c67be3d5b02dcbd86c2', 'request_id': 'req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.133825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] The token cache is being invalidate because project 7ed32bd8cf964c67be3d5b02dcbd86c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:20.134516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.135905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.138080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 971/1939] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:19 2022] DELETE /identity/v3/projects/7ed32bd8cf964c67be3d5b02dcbd86c2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:20.146181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.147883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.207768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.207768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.207768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.207768 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.209583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.209583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.215376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:20.241722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 969/1940] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:20.246863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.248825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.260655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f42da2-1e4b-41f1-acaf-fe3e40aa48e5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.280559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:20.314822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199790ab-1239-49e1-bf36-8a9eba126567 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.320212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.320704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.320986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.321251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.321882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.323808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.326343 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.328181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.328462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.329311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] PATH_INFO: `/v3/users/ab638d0d58134112a96eb72a9904eadb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.330574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab638d0d58134112a96eb72a9904eadb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.332652 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.351525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 970/1941] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:20.354896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.367840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1013-fca2-4ee0-85ff-f1023aaa1098 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.375365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.376528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.376810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.377078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.377738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.379355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.384879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab638d0d58134112a96eb72a9904eadb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.390000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 971/1942] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:20.394058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab638d0d58134112a96eb72a9904eadb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.394720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab638d0d58134112a96eb72a9904eadb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.395185 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.395804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab638d0d58134112a96eb72a9904eadb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.397088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.404484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab638d0d58134112a96eb72a9904eadb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.415466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b6177-4802-4bda-a5a6-1540be643d23 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.420673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8c1ff-9dab-4885-8db9-36a659939606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.421297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c1ff-9dab-4885-8db9-36a659939606 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.421585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c1ff-9dab-4885-8db9-36a659939606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.421836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c1ff-9dab-4885-8db9-36a659939606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.422508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c1ff-9dab-4885-8db9-36a659939606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.423808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 972/1943] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:20 2022] DELETE /identity/v3/users/ab638d0d58134112a96eb72a9904eadb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:20.424287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c1ff-9dab-4885-8db9-36a659939606 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.441390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 972/1944] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:20.442512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.533552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9943-3cf2-40bb-9ce6-3277f6162476 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.538145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.538802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.539077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.539346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] PATH_INFO: `/v3/projects/3a33f435b74f4f3bbdd2acb5a298cd4b/users/58e2bdbdba234db095879d428461a596/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.554303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a33f435b74f4f3bbdd2acb5a298cd4b, user_id=58e2bdbdba234db095879d428461a596, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.557390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.579600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 973/1945] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:20 2022] PUT /identity/v3/projects/3a33f435b74f4f3bbdd2acb5a298cd4b/users/58e2bdbdba234db095879d428461a596/roles/87f1831257824560813f954348534540 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:20.584448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f03582-edde-43cf-a516-354659c630af None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.584740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f03582-edde-43cf-a516-354659c630af None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.585024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f03582-edde-43cf-a516-354659c630af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.629996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b1f03582-edde-43cf-a516-354659c630af None None] MFA Rules not processed for user `58e2bdbdba234db095879d428461a596`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:20.631881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b1f03582-edde-43cf-a516-354659c630af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.715847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.718071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.719198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 973/1946] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:20.724366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f03582-edde-43cf-a516-354659c630af None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.772788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:20.773105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:20.793864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f03582-edde-43cf-a516-354659c630af None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.799813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.800446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.800721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.801016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.801644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.803545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.810792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7eee-4a80-4bc3-b9f2-e15203b3629e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.816491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.817123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.817403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.817740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] PATH_INFO: `/v3/projects/714ff544a3db484abf71c9f97f2bc0e2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.818927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] RBAC: Authorizing `identity:delete_project(project_id=714ff544a3db484abf71c9f97f2bc0e2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.821241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.832159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 974/1947] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:20.836266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.873449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714ff544a3db484abf71c9f97f2bc0e2', 'request_id': 'req-746ed61c-3c5f-4466-be2b-9395488a601d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.879852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714ff544a3db484abf71c9f97f2bc0e2', 'request_id': 'req-746ed61c-3c5f-4466-be2b-9395488a601d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.887971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714ff544a3db484abf71c9f97f2bc0e2', 'request_id': 'req-746ed61c-3c5f-4466-be2b-9395488a601d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.899793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5971-4d9a-4091-8bc8-7dbdbe2c0f5c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:20.900015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] The token cache is being invalidate because project 714ff544a3db484abf71c9f97f2bc0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:20.900345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.901106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-746ed61c-3c5f-4466-be2b-9395488a601d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:20.902736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 974/1948] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:20 2022] DELETE /identity/v3/projects/714ff544a3db484abf71c9f97f2bc0e2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:20.907615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1b82e-dd5a-4ac4-a5f1-dad3a4918da0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.907911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1b82e-dd5a-4ac4-a5f1-dad3a4918da0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.908189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1b82e-dd5a-4ac4-a5f1-dad3a4918da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.928533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d9f1b82e-dd5a-4ac4-a5f1-dad3a4918da0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:20.929727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d9f1b82e-dd5a-4ac4-a5f1-dad3a4918da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.957071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:20.957921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.958201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.958470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.959088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:20.961024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:20.984492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 975/1949] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:20.986227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:20.987849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:20.988283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 975/1950] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:20.996055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:20.996349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:20.996627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:20.999603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df0522-475b-47e2-a37f-1f8a652c0a4d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.004878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.005473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.006002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.006270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.006919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.008517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.017110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:21.018308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:21.022523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 976/1951] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:21.023601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.026419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 976/1952] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:21.030505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.033816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d4ce-7f86-406b-965b-4de25edf01d6 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.038564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.039208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.039503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.039916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] PATH_INFO: `/v3/projects/fa52ec0d0e224054a34ebfd42c1ee953/users/62b3cf73bb3f4a93b42d4207e2303d0b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.040114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c78f-cb30-4c6a-9ac1-f1d686c3efee None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.044973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.045561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.045840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.046113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.047969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:21.048395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.050025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.055852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa52ec0d0e224054a34ebfd42c1ee953, user_id=62b3cf73bb3f4a93b42d4207e2303d0b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.058613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.062068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 977/1953] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:45:21.067823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.072578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 977/1954] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:21 2022] PUT /identity/v3/projects/fa52ec0d0e224054a34ebfd42c1ee953/users/62b3cf73bb3f4a93b42d4207e2303d0b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:21.076524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.077659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb7f7-5ebf-444f-988e-e1e8ba53ca13 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.111884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:21.112807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:21.141140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.141792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.142074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.142394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.142967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.144930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.149086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f493e-317b-40aa-84e3-245fa4c23d16 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.155334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.156178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.156468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.157401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.158064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.159849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.173977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 978/1955] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:21.187507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 978/1956] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:21 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:21.187507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.187906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.191070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b54e-0b81-4616-a41c-f4e5d1be10df admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.260552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8f6b-3017-4e94-b9b9-4934db7e0c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.261275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8f6b-3017-4e94-b9b9-4934db7e0c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.261412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5343f30-63cb-468e-8114-4bf1571d21dc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.261769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8f6b-3017-4e94-b9b9-4934db7e0c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.261844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8f6b-3017-4e94-b9b9-4934db7e0c24 admin admin] PATH_INFO: `/v3/projects/fa52ec0d0e224054a34ebfd42c1ee953/users/62b3cf73bb3f4a93b42d4207e2303d0b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.264342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b8f6b-3017-4e94-b9b9-4934db7e0c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa52ec0d0e224054a34ebfd42c1ee953, user_id=62b3cf73bb3f4a93b42d4207e2303d0b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.266645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.267069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b8f6b-3017-4e94-b9b9-4934db7e0c24 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.267578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.267860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.268220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.268855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.271636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.282933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 979/1957] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:21 2022] PUT /identity/v3/projects/fa52ec0d0e224054a34ebfd42c1ee953/users/62b3cf73bb3f4a93b42d4207e2303d0b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:21.293449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.293782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.294056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.302577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 979/1958] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:21 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:21.306525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.318725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9744db5-d02d-4379-9122-0863246731b5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.337764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] MFA Rules not processed for user `62b3cf73bb3f4a93b42d4207e2303d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:21.339749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:21.423937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.424693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.424975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.425252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.426118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.427644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.439757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 980/1959] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:21.440013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 980/1960] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:21.443442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.450574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.452227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:21.457996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963c4cc-1dc9-4b71-aa7c-1c4eea501bf6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.464328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.464906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.465217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.465521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.466167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.467801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.481243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 981/1961] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:21.485727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.497368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7577a-e434-4da0-869b-7b28bf221771 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.502020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a345d-0f43-497d-b82d-6e2ba0850b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.502667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a345d-0f43-497d-b82d-6e2ba0850b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.502944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a345d-0f43-497d-b82d-6e2ba0850b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.503216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a345d-0f43-497d-b82d-6e2ba0850b28 admin admin] PATH_INFO: `/v3/projects/247725b545e54c8b9228def49052b7c5/users/1bbd24b973184b0d835f15b877b7760a/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.515305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:21.522027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce50fd-6bb4-4b5f-ae6e-415a850fd983 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.522137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a345d-0f43-497d-b82d-6e2ba0850b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=247725b545e54c8b9228def49052b7c5, user_id=1bbd24b973184b0d835f15b877b7760a, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.526032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a345d-0f43-497d-b82d-6e2ba0850b28 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.527518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.528337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.528794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.529276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.541782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.543265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.543545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 982/1962] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:21 2022] PUT /identity/v3/projects/247725b545e54c8b9228def49052b7c5/users/1bbd24b973184b0d835f15b877b7760a/roles/87f1831257824560813f954348534540 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:21.553666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 981/1963] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:21.554066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.554362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.554642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.580830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:21.585247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] MFA Rules not processed for user `1bbd24b973184b0d835f15b877b7760a`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:21.586584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:21.669994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 983/1964] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:21.674698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.691961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aa044-69b5-46bd-b23d-d6f9d328801d service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.750895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.751539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.751818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.752088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.752709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.754629 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.792037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 982/1965] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:21 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:21.796279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.810074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61589c-f621-4f4a-875d-6fffc170f113 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.892021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.892642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.892923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.893197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.893842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.895835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.922796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 984/1966] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:21 2022] POST /identity/v3/users => generated 301 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:21.924248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.935115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8f89-85d0-41ae-8956-3bc0c5344bde admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.942469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.943146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.948476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.948476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:21.948476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:21.948476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:21.961312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 983/1967] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:21.966022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:21.976041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec2226-d96b-4249-8996-61b3b1de456c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:21.983865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:21.984544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:21.984992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:21.985275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] PATH_INFO: `/v3/projects/6e059a0273ba47e18f63a0dc215804df/users/8fdebea35aa643348c78daeb0b577a19/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.001232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e059a0273ba47e18f63a0dc215804df, user_id=8fdebea35aa643348c78daeb0b577a19, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:22.003959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:22.020320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 985/1968] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:21 2022] PUT /identity/v3/projects/6e059a0273ba47e18f63a0dc215804df/users/8fdebea35aa643348c78daeb0b577a19/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:22.024153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:22.034156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2a306-e677-4c33-a175-4f63833ab7e8 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:22.101144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b61ba2-521b-454b-a806-cf49d832db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:22.101856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b61ba2-521b-454b-a806-cf49d832db77 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.102129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b61ba2-521b-454b-a806-cf49d832db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.102395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b61ba2-521b-454b-a806-cf49d832db77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.103062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b61ba2-521b-454b-a806-cf49d832db77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:22.104648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b61ba2-521b-454b-a806-cf49d832db77 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:22.118447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 984/1969] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:22.119543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:22.130256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b6f35-a03f-4031-b22c-94b891d37a2c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:22.135218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:22.135894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.136178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.136449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] PATH_INFO: `/v3/projects/6e059a0273ba47e18f63a0dc215804df/users/8fdebea35aa643348c78daeb0b577a19/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.138355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e059a0273ba47e18f63a0dc215804df, user_id=8fdebea35aa643348c78daeb0b577a19, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:22.141066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:22.151989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 986/1970] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:22 2022] PUT /identity/v3/projects/6e059a0273ba47e18f63a0dc215804df/users/8fdebea35aa643348c78daeb0b577a19/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:22.163405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.163731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.164011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.194333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] MFA Rules not processed for user `8fdebea35aa643348c78daeb0b577a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:22.195592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:22.254707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:22.300767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 985/1971] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:22.311101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:22.312829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:22.373760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f6811-43e9-4e9d-94f8-506568d5e582 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:22.378406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f7098-67a8-4394-8afb-a459e1de0419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:22.379020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f7098-67a8-4394-8afb-a459e1de0419 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.379322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f7098-67a8-4394-8afb-a459e1de0419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.379592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f7098-67a8-4394-8afb-a459e1de0419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.389806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f7098-67a8-4394-8afb-a459e1de0419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:22.391302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f7098-67a8-4394-8afb-a459e1de0419 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:22.395656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:22.413865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 987/1972] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:22.565286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:22.679539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db3f7e-344c-4486-ac66-e2b40d6e89df None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:22.679539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:22.679539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.679539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.681049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] PATH_INFO: `/v3/users/1bbd24b973184b0d835f15b877b7760a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.681049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bbd24b973184b0d835f15b877b7760a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:22.682807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:22.735347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bbd24b973184b0d835f15b877b7760a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:22.743283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bbd24b973184b0d835f15b877b7760a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:22.743978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bbd24b973184b0d835f15b877b7760a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:22.751233 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bbd24b973184b0d835f15b877b7760a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:22.751878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bbd24b973184b0d835f15b877b7760a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:22.774214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 986/1973] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:22 2022] DELETE /identity/v3/users/1bbd24b973184b0d835f15b877b7760a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:22.778370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.778861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.779346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.801473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:22.802771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:22.885291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 988/1974] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:22.897076 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:22.901075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:22.966324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b2de9-4257-4943-9211-d9e556189131 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:22.971008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9658-3669-485e-9ca9-8073e13e16c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:22.971698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9658-3669-485e-9ca9-8073e13e16c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:22.971934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9658-3669-485e-9ca9-8073e13e16c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:22.972288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9658-3669-485e-9ca9-8073e13e16c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:22.986655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9658-3669-485e-9ca9-8073e13e16c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:22.988111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9658-3669-485e-9ca9-8073e13e16c3 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:23.002657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 987/1975] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:23.161123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:23.162783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:23.179969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8d4aa-d698-41fe-8929-35f2736f53b6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:23.186701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:23.187682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:23.188484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:23.188770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] PATH_INFO: `/v3/users/58e2bdbdba234db095879d428461a596` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:23.191086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e2bdbdba234db095879d428461a596)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:23.194490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:23.261622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e2bdbdba234db095879d428461a596'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.270739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e2bdbdba234db095879d428461a596'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.271884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e2bdbdba234db095879d428461a596'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.272608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e2bdbdba234db095879d428461a596'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.283164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e2bdbdba234db095879d428461a596'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.354487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 989/1976] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:23 2022] DELETE /identity/v3/users/58e2bdbdba234db095879d428461a596 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:23.356995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:23.358091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:23.358621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:23.382187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:23.393647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:23.398161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:23.474874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:23.476954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:23.512535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 988/1977] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:23.524463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:23.526064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:23.625877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf854e2-22e3-4cc9-9e7b-c77b1710bda5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:23.638479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:23.639221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:23.639614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:23.639891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] PATH_INFO: `/v3/projects/247725b545e54c8b9228def49052b7c5` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:23.644484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=247725b545e54c8b9228def49052b7c5)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:23.647754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:23.650575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e90d04-10b9-4010-9e87-53671f1f2aa5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:23.654695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:23.655385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:23.655778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:23.656201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:23.670320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:23.671921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:23.674415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:23.691471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 989/1978] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:23.769148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247725b545e54c8b9228def49052b7c5', 'request_id': 'req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.785189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247725b545e54c8b9228def49052b7c5', 'request_id': 'req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.798035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247725b545e54c8b9228def49052b7c5', 'request_id': 'req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.814698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] The token cache is being invalidate because project 247725b545e54c8b9228def49052b7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:23.814698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.815663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:23.818221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 990/1979] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:23 2022] DELETE /identity/v3/projects/247725b545e54c8b9228def49052b7c5 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:23.822079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:23.834266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca3cf7-d919-45d8-abef-fe074d170f82 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:23.912146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:23.913670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:23.914015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:23.914300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] PATH_INFO: `/v3/users/8fdebea35aa643348c78daeb0b577a19` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:23.916111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdebea35aa643348c78daeb0b577a19)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:23.918814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:24.021080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdebea35aa643348c78daeb0b577a19'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.027773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:24.032718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdebea35aa643348c78daeb0b577a19'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.035296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdebea35aa643348c78daeb0b577a19'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.049911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdebea35aa643348c78daeb0b577a19'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.051029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdebea35aa643348c78daeb0b577a19'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.072782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 990/1980] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:23 2022] DELETE /identity/v3/users/8fdebea35aa643348c78daeb0b577a19 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:24.105287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:24.106852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.120644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b85d-1da2-4668-b083-8a66a4e6e8c0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:24.196882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:24.197860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.198142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.198449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] PATH_INFO: `/v3/projects/3a33f435b74f4f3bbdd2acb5a298cd4b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.199875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a33f435b74f4f3bbdd2acb5a298cd4b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:24.201882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:24.257513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a33f435b74f4f3bbdd2acb5a298cd4b', 'request_id': 'req-f8714375-a0ed-4b3f-94ff-bb5419ed75da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.266593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a33f435b74f4f3bbdd2acb5a298cd4b', 'request_id': 'req-f8714375-a0ed-4b3f-94ff-bb5419ed75da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.277332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a33f435b74f4f3bbdd2acb5a298cd4b', 'request_id': 'req-f8714375-a0ed-4b3f-94ff-bb5419ed75da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.292024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] The token cache is being invalidate because project 3a33f435b74f4f3bbdd2acb5a298cd4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:24.292152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.293278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.294691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 991/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:24 2022] DELETE /identity/v3/projects/3a33f435b74f4f3bbdd2acb5a298cd4b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:24.299064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:24.300626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.314939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3e5c-48fc-486c-abe8-0f0afbf82b6d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:24.387896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:24.388607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.388882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.389617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] PATH_INFO: `/v3/users/62b3cf73bb3f4a93b42d4207e2303d0b` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.391752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b3cf73bb3f4a93b42d4207e2303d0b)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:24.393859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:24.449297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b3cf73bb3f4a93b42d4207e2303d0b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.459590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b3cf73bb3f4a93b42d4207e2303d0b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.460396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b3cf73bb3f4a93b42d4207e2303d0b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.470212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b3cf73bb3f4a93b42d4207e2303d0b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.471050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-4fd46aa8-d9a8-4e98-9d9a-f2a6530416d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b3cf73bb3f4a93b42d4207e2303d0b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.494124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 991/1982] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:24 2022] DELETE /identity/v3/users/62b3cf73bb3f4a93b42d4207e2303d0b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:24.515709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:24.525629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.533725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe248896-ff21-4dae-87ab-c6469876f429 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.534321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe248896-ff21-4dae-87ab-c6469876f429 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.534321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe248896-ff21-4dae-87ab-c6469876f429 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.536042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 992/1983] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:45:24.544494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714375-a0ed-4b3f-94ff-bb5419ed75da admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:24.550229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.550527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.550932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.596806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:24.598628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:24.600984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.643796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:24.644543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.644882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.645229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] PATH_INFO: `/v3/projects/6e059a0273ba47e18f63a0dc215804df` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.646452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e059a0273ba47e18f63a0dc215804df)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:24.648486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:24.681241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:24.716449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 993/1984] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:45:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:24.732849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:24.734850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.745711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e059a0273ba47e18f63a0dc215804df', 'request_id': 'req-ea777651-2e89-4fc7-b09c-b1e136e5c256'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.759767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb8a0e-f1df-4738-bdd7-0b2aa2890ca1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:24.771607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e059a0273ba47e18f63a0dc215804df', 'request_id': 'req-ea777651-2e89-4fc7-b09c-b1e136e5c256'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.798984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e059a0273ba47e18f63a0dc215804df', 'request_id': 'req-ea777651-2e89-4fc7-b09c-b1e136e5c256'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.840420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] The token cache is being invalidate because project 6e059a0273ba47e18f63a0dc215804df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:24.840854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.841993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea777651-2e89-4fc7-b09c-b1e136e5c256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:24.844038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 992/1985] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:24 2022] DELETE /identity/v3/projects/6e059a0273ba47e18f63a0dc215804df => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:24.850269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471554b9-3c7f-447d-800a-6ab52e2eecab None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.850624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471554b9-3c7f-447d-800a-6ab52e2eecab None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.850974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471554b9-3c7f-447d-800a-6ab52e2eecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.892808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-471554b9-3c7f-447d-800a-6ab52e2eecab None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:24.894359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-471554b9-3c7f-447d-800a-6ab52e2eecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.913637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:24.914517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:24.914959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:24.915454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:24.919926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.934227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:24.936036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:24.948256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 994/1986] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:24.948917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:24.950294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:24.996471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 993/1987] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:25.004871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944546fe-33db-439f-8572-b7a053914cdf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.005156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944546fe-33db-439f-8572-b7a053914cdf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.005432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944546fe-33db-439f-8572-b7a053914cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.041449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-944546fe-33db-439f-8572-b7a053914cdf None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:25.042665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-944546fe-33db-439f-8572-b7a053914cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:25.056222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 994/1988] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:25.060987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944546fe-33db-439f-8572-b7a053914cdf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.073291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688b8f2-2e5c-4a60-94fe-eab7370311c2 service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.076678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944546fe-33db-439f-8572-b7a053914cdf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.082925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.083871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.083871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.084130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.084883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:25.086848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.087619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.087907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.088182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] PATH_INFO: `/v3/projects/fa52ec0d0e224054a34ebfd42c1ee953` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.088436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.090584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.092206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa52ec0d0e224054a34ebfd42c1ee953)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.097124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.116804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 995/1989] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:25.126016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.154583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394d2c-74e0-4c88-84d3-dda66556f760 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.165686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.166389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.166674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.167358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.168690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.170916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.187811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa52ec0d0e224054a34ebfd42c1ee953', 'request_id': 'req-d61bd5d9-5363-469e-9078-fc51cea8c896'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:25.197643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa52ec0d0e224054a34ebfd42c1ee953', 'request_id': 'req-d61bd5d9-5363-469e-9078-fc51cea8c896'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:25.211070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa52ec0d0e224054a34ebfd42c1ee953', 'request_id': 'req-d61bd5d9-5363-469e-9078-fc51cea8c896'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:25.214008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:25.236630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 996/1990] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:25.240312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.245948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] The token cache is being invalidate because project fa52ec0d0e224054a34ebfd42c1ee953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:25.247935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:25.251057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d61bd5d9-5363-469e-9078-fc51cea8c896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:25.256240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 995/1991] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:24 2022] DELETE /identity/v3/projects/fa52ec0d0e224054a34ebfd42c1ee953 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:25.261823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c486897-74ee-4751-80ff-39b4ada12beb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.275250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:25.276100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c486897-74ee-4751-80ff-39b4ada12beb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.278894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c486897-74ee-4751-80ff-39b4ada12beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.318161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7c486897-74ee-4751-80ff-39b4ada12beb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:25.319758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7c486897-74ee-4751-80ff-39b4ada12beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:25.350471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2d43-a645-446e-b55a-544c0de59cd2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.359521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.361023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.361601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.362072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.363152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.366768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.402276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 997/1992] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:25.403948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.406948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:25.421207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5775f25-d88e-4c40-a56c-2deeceb71cac admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.427276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.428048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.428422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.428759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.429475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.431264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.432387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 996/1993] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:25.440450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.440749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.441073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.446521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 998/1994] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:25.447727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.461176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949839a-c86e-4af5-8cf8-194fc6df771d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.461308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:25.462544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:25.475523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.475523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.475523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.475523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.475893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.475963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 997/1995] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:25.476048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.476119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.485782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feb325-2d4b-4363-8892-611a47699a7b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.489173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 999/1996] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:25.489960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.490454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.491053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.491362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.491667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.492321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:25.492816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.494477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.501258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa28ec-1a60-4c1e-a1f0-eb1d4d628fe1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.507200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350de59e-148f-4ad0-adb2-9ce91aec9086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.507969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350de59e-148f-4ad0-adb2-9ce91aec9086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.508247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350de59e-148f-4ad0-adb2-9ce91aec9086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.508514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350de59e-148f-4ad0-adb2-9ce91aec9086 admin admin] PATH_INFO: `/v3/projects/bf07b096bc4e4bc3883109c6c983e07b/users/84ee4366fb9245d68b1274e0649160b2/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.511523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 998/1997] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:45:25.514908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.526355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a7cc-079b-4ee4-bdde-971801cb218a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.527444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350de59e-148f-4ad0-adb2-9ce91aec9086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf07b096bc4e4bc3883109c6c983e07b, user_id=84ee4366fb9245d68b1274e0649160b2, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.530223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350de59e-148f-4ad0-adb2-9ce91aec9086 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.534349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.535335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.535939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.536358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.537343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.540486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.546879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1000/1998] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:25 2022] PUT /identity/v3/projects/bf07b096bc4e4bc3883109c6c983e07b/users/84ee4366fb9245d68b1274e0649160b2/roles/87f1831257824560813f954348534540 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:25.558002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.558284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.558557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.578021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 999/1999] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:25.582162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.591132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] MFA Rules not processed for user `84ee4366fb9245d68b1274e0649160b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:25.592623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:25.663368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9201ca-a471-42aa-b009-45a639ff511e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.674653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.675335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.675647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.675921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.676564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.678832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.683818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1001/2000] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:25.690778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.706345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1000/2001] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:25.706590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e493c1-9115-42d3-9f66-dd826616ec4b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.710364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.721317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.722198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.722573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.722900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.723615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.724714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add14a49-03f1-4815-9723-200c095bf223 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.725522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.729677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.730291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.730526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.730788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.731449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.733143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.747788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1001/2002] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:25.749100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.754165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1002/2003] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:25.758275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.760574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353bd96-f410-4994-9aa2-c1727a78f058 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.790453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:25.792861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:25.818649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.819252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.819563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.819836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.820529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.822134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.823861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b380-103b-4e86-9157-29c6b7195cea admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.828684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.829326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.829737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.830170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.831211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.832106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1002/2004] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:25.834629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.836364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.850860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fdd72-07b0-4d06-8dca-098708bb6685 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.856082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.856920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.857457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.857739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] PATH_INFO: `/v3/projects/a91ae1f7a6b54543927220c058e184a7/users/34dd63dc1c314f2e8fd5a3ae9b6e9c67/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.863842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1003/2005] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:25.867554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.875924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91ae1f7a6b54543927220c058e184a7, user_id=34dd63dc1c314f2e8fd5a3ae9b6e9c67, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.877824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17717afa-a7ea-4185-bf3f-f3954cf9a63e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.878576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.882928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60211545-1c68-4046-ae05-25fb8f370cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.883509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60211545-1c68-4046-ae05-25fb8f370cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.883992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60211545-1c68-4046-ae05-25fb8f370cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.884221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60211545-1c68-4046-ae05-25fb8f370cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.884897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60211545-1c68-4046-ae05-25fb8f370cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:25.886450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60211545-1c68-4046-ae05-25fb8f370cf8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:25.899532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1003/2006] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:25 2022] PUT /identity/v3/projects/a91ae1f7a6b54543927220c058e184a7/users/34dd63dc1c314f2e8fd5a3ae9b6e9c67/roles/87f1831257824560813f954348534540 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:25.900650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1004/2007] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:25.901621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:25.910918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.910918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.910918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:25.938051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] MFA Rules not processed for user `34dd63dc1c314f2e8fd5a3ae9b6e9c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:25.939380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:25.983218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805feb27-1cc5-43f1-9718-b76b7f91caaf admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:25.988143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:25.988793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:25.989071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:25.989341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] PATH_INFO: `/v3/projects/b776df921106476c9e95321f8c6c1e59/users/c445cc781c2449b282c6796093564c89/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.003262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b776df921106476c9e95321f8c6c1e59, user_id=c445cc781c2449b282c6796093564c89, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.006100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.007623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1005/2008] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:26.012158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.020582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1004/2009] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:25 2022] PUT /identity/v3/projects/b776df921106476c9e95321f8c6c1e59/users/c445cc781c2449b282c6796093564c89/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:26.023826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9462eae-6006-4bc1-82b0-9f42069ef827 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.024435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.080935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.081540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.081819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.082117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.082778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.084712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.087239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115809b1-5781-4130-92c4-9f13ebd5865a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.091952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.092549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.092827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.093097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.093739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.096033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.106821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1005/2010] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:26.111025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.111707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1006/2011] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:26 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:26.115528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.123198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123747b2-bc7b-413e-9819-9d4192e3ce66 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.143546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:26.145810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:26.180202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673614a-dc0d-4c86-a331-9e1a2b74aeec admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.184944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.185093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e57f0-a5c8-4824-91c3-8cfe0fc2f3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.185541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.185650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e57f0-a5c8-4824-91c3-8cfe0fc2f3c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.185814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.185924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e57f0-a5c8-4824-91c3-8cfe0fc2f3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.186083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.186193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e57f0-a5c8-4824-91c3-8cfe0fc2f3c9 admin admin] PATH_INFO: `/v3/projects/b776df921106476c9e95321f8c6c1e59/users/c445cc781c2449b282c6796093564c89/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.186699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.188140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e57f0-a5c8-4824-91c3-8cfe0fc2f3c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b776df921106476c9e95321f8c6c1e59, user_id=c445cc781c2449b282c6796093564c89, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.188682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.190759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e57f0-a5c8-4824-91c3-8cfe0fc2f3c9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.200885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1006/2012] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:26 2022] PUT /identity/v3/projects/b776df921106476c9e95321f8c6c1e59/users/c445cc781c2449b282c6796093564c89/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:26.211829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1007/2013] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:26 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:26.211919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.211996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.212261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.215265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.227629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cbdb1-96e2-4546-a8d3-1e71b95397d8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.244894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] MFA Rules not processed for user `c445cc781c2449b282c6796093564c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:26.246827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:26.293621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.295050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.295338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.295647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.296450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.298481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.315572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1008/2014] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:26.317724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.332243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbfe2a-bddf-4eb4-aa2a-e707df0baafb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.339092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.340202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.340703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.341170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] PATH_INFO: `/v3/projects/4529b1d98bb94ee7a0ce691312b2599e/users/815042d616d04bb7b10245defcecd620/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.371760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1007/2015] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:26.373011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4529b1d98bb94ee7a0ce691312b2599e, user_id=815042d616d04bb7b10245defcecd620, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.376610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.386708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.389198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:26.404672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1009/2016] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:26 2022] PUT /identity/v3/projects/4529b1d98bb94ee7a0ce691312b2599e/users/815042d616d04bb7b10245defcecd620/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:26.405694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.462355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5a02d-eb7f-4dea-a448-6d57cc105f7d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.486108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519acd5e-1fb7-43e0-ba52-487813bdf70c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.493100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.493863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.494267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.494960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.495373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.497993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.515642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1010/2017] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:26.521695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.532525 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafb615-bf29-4088-b476-b3394be9c236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.533098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafb615-bf29-4088-b476-b3394be9c236 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.533384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafb615-bf29-4088-b476-b3394be9c236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.533597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafb615-bf29-4088-b476-b3394be9c236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.540321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283a42f-f28b-4a4e-92eb-f7955fb270f8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.549285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.550312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.550906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.552597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] PATH_INFO: `/v3/projects/4529b1d98bb94ee7a0ce691312b2599e/users/815042d616d04bb7b10245defcecd620/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.553869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:26.556216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4529b1d98bb94ee7a0ce691312b2599e, user_id=815042d616d04bb7b10245defcecd620, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.560579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.610375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1011/2018] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:26 2022] PUT /identity/v3/projects/4529b1d98bb94ee7a0ce691312b2599e/users/815042d616d04bb7b10245defcecd620/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:26.610375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.613644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:26.632034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:26.643789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafb615-bf29-4088-b476-b3394be9c236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.645957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafb615-bf29-4088-b476-b3394be9c236 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.657660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1008/2019] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:26.659144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.659436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.659899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.702463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] MFA Rules not processed for user `815042d616d04bb7b10245defcecd620`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:26.704212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:26.738995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab2003-bf61-40b6-9b0b-0f3a5ecba521 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:26.745506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:26.746690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.747234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.747740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] PATH_INFO: `/v3/users/47a8e8c2c44b4908bd7c4b00a9ff1487` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.749452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a8e8c2c44b4908bd7c4b00a9ff1487)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:26.752945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:26.832822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a8e8c2c44b4908bd7c4b00a9ff1487'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:26.844400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a8e8c2c44b4908bd7c4b00a9ff1487'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:26.845590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a8e8c2c44b4908bd7c4b00a9ff1487'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:26.846667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a8e8c2c44b4908bd7c4b00a9ff1487'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:26.864543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1009/2020] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:26.869584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a71648c7-b9b4-4f9a-b866-875e8bc93ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a8e8c2c44b4908bd7c4b00a9ff1487'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:26.888771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:26.890364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:26.899689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1012/2021] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:26 2022] DELETE /identity/v3/users/47a8e8c2c44b4908bd7c4b00a9ff1487 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:26.904357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:26.904766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:26.905076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:26.927475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:26.929186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:26.982059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eab192-976b-4c62-9925-ef5509792b39 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:27.016085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1013/2022] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:27.024704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:27.026404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.068388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:27.069619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:27.070463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:27.070809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:27.080384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.092037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:27.095842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:27.123061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f1787-3408-4b9d-bd6b-01cb65ee7757 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:27.129270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:27.129882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:27.130454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:27.130990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:27.143976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:27.145818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:27.159534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1014/2023] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:27.177933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:27.180283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:27.199988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1010/2024] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:27.494628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:27.496163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.520179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6a48-dd0b-4b8d-925e-5e3aa480918b service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:27.525675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:27.526320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:27.526572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:27.526842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] PATH_INFO: `/v3/projects/fa4391c518004195b3f62c6d622f8f28` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:27.528022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4391c518004195b3f62c6d622f8f28)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:27.530129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:27.587579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4391c518004195b3f62c6d622f8f28', 'request_id': 'req-4ef72593-38b9-464a-a204-5307eaae68c8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.596241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4391c518004195b3f62c6d622f8f28', 'request_id': 'req-4ef72593-38b9-464a-a204-5307eaae68c8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.613630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4391c518004195b3f62c6d622f8f28', 'request_id': 'req-4ef72593-38b9-464a-a204-5307eaae68c8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.614008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:27.615424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.629496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] The token cache is being invalidate because project fa4391c518004195b3f62c6d622f8f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:27.629827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.630491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.632497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1015/2025] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:27 2022] DELETE /identity/v3/projects/fa4391c518004195b3f62c6d622f8f28 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:27.633402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372b7a0-f086-495d-afb2-133b9ec05880 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:27.638631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:27.642074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.704530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:27.705135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:27.705411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:27.705679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] PATH_INFO: `/v3/users/84ee4366fb9245d68b1274e0649160b2` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:27.706885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ee4366fb9245d68b1274e0649160b2)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:27.708820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:27.732267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef72593-38b9-464a-a204-5307eaae68c8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:27.736890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:27.737510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:27.737785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:27.738053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] PATH_INFO: `/v3/users/9299a804af8a40fd9cad8a304f50f660` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:27.739165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9299a804af8a40fd9cad8a304f50f660)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:27.741227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:27.768265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ee4366fb9245d68b1274e0649160b2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.779399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ee4366fb9245d68b1274e0649160b2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.780597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ee4366fb9245d68b1274e0649160b2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.790606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ee4366fb9245d68b1274e0649160b2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.791325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-baf4df5a-892d-4ebf-a1d1-7b291ec8d682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ee4366fb9245d68b1274e0649160b2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.792749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9299a804af8a40fd9cad8a304f50f660'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.800671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9299a804af8a40fd9cad8a304f50f660'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.801435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9299a804af8a40fd9cad8a304f50f660'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.802268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9299a804af8a40fd9cad8a304f50f660'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.804771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:27.814190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9299a804af8a40fd9cad8a304f50f660'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:27.816462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1011/2026] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:27 2022] DELETE /identity/v3/users/84ee4366fb9245d68b1274e0649160b2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:27.820742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:27.821163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:27.821443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:27.838431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1016/2027] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:27 2022] DELETE /identity/v3/users/9299a804af8a40fd9cad8a304f50f660 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:27.842356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:27.843590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.867081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:27.922256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1012/2028] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:27.932266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:27.933705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:27.996399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fb966-05ba-4bb3-8c12-8b0a37f92a59 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:28.001143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:28.001780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.002047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.002320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.013557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:28.015688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:28.027744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1017/2029] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:28.546168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:28.548373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:28.560777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59091f38-66a9-4aea-aa40-644cd9fd3273 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:28.565417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:28.566051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.566331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.566694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] PATH_INFO: `/v3/projects/fdbfd7c131134438a6dd37d6c6407113` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.567798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbfd7c131134438a6dd37d6c6407113)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:28.569697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:28.620456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbfd7c131134438a6dd37d6c6407113', 'request_id': 'req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.628985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbfd7c131134438a6dd37d6c6407113', 'request_id': 'req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.632389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:28.634405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:28.637164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbfd7c131134438a6dd37d6c6407113', 'request_id': 'req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.653497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] The token cache is being invalidate because project fdbfd7c131134438a6dd37d6c6407113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:28.653786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.654443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e105ae2e-30bf-4360-bbe3-5d4a2c95131e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.656023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1013/2030] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:28 2022] DELETE /identity/v3/projects/fdbfd7c131134438a6dd37d6c6407113 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:28.662637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7852b4-ec49-49a7-a432-ac2c7c9aeb71 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.662914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7852b4-ec49-49a7-a432-ac2c7c9aeb71 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.663375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7852b4-ec49-49a7-a432-ac2c7c9aeb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.680029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f369-6980-424f-b7a5-f5dd2ec1a580 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:28.685907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7e7852b4-ec49-49a7-a432-ac2c7c9aeb71 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:28.687258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7e7852b4-ec49-49a7-a432-ac2c7c9aeb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:28.752963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:28.753599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.753873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.754149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] PATH_INFO: `/v3/projects/bf07b096bc4e4bc3883109c6c983e07b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.755357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf07b096bc4e4bc3883109c6c983e07b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:28.757551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:28.759061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1014/2031] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:28.766904 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.767234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.767535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.790381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:28.791630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:28.805833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1015/2032] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:28.807046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:28.817188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4b813-1531-4b07-a4ba-0ac59ffd97e3 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:28.827679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf07b096bc4e4bc3883109c6c983e07b', 'request_id': 'req-a14299c2-9704-4427-9187-cb0baf3e6922'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.834358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf07b096bc4e4bc3883109c6c983e07b', 'request_id': 'req-a14299c2-9704-4427-9187-cb0baf3e6922'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.841758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf07b096bc4e4bc3883109c6c983e07b', 'request_id': 'req-a14299c2-9704-4427-9187-cb0baf3e6922'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.854441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] The token cache is being invalidate because project bf07b096bc4e4bc3883109c6c983e07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:28.854735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.855387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:28.857101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1018/2033] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:28 2022] DELETE /identity/v3/projects/bf07b096bc4e4bc3883109c6c983e07b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:28.861397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:28.862739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:28.866502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:28.874751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:28.892430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:28.893041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.894002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.894002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.894522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:28.894822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:28.896503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:28.909898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1016/2034] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:28.914173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:28.915983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:28.928229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8802e02-1b29-4de2-ae3d-5cd8bd23550e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:28.932747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:28.933329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.933771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.933879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.934680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:28.936764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:28.938956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14299c2-9704-4427-9187-cb0baf3e6922 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:28.943890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:28.944451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:28.944833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:28.945107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] PATH_INFO: `/v3/users/c445cc781c2449b282c6796093564c89` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:28.946242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c445cc781c2449b282c6796093564c89)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:28.948189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:28.966264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1017/2035] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:28 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:28.971218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:28.997475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c445cc781c2449b282c6796093564c89'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.005145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c445cc781c2449b282c6796093564c89'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.006572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c445cc781c2449b282c6796093564c89'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.006687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c445cc781c2449b282c6796093564c89'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.013880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c445cc781c2449b282c6796093564c89'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.032226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1019/2036] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:28 2022] DELETE /identity/v3/users/c445cc781c2449b282c6796093564c89 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:29.036521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37544ced-2a84-451b-bd79-68938ab71306 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.094805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.095412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.097259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.097846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.098158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.100295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.124172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1018/2037] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:28 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:29.128438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.129844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.144072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6bc53-dba3-4438-ab4c-b4ddf49ed67a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.148979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.149836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.149836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.150291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.151247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.152846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.163794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1020/2038] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:29.167755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.177853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd9b6-e4d0-4c8e-acc8-fc2fa2cd5f06 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.183389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed74e50-e0e0-4f2c-a433-d55b15d4f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.184037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed74e50-e0e0-4f2c-a433-d55b15d4f623 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.184316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed74e50-e0e0-4f2c-a433-d55b15d4f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.184591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed74e50-e0e0-4f2c-a433-d55b15d4f623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.185247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed74e50-e0e0-4f2c-a433-d55b15d4f623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.186925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed74e50-e0e0-4f2c-a433-d55b15d4f623 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.197862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1019/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:29.201912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.212655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f708230-4b60-425f-af80-ebe286526698 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.217371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.218024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.218304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.218581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] PATH_INFO: `/v3/projects/7028ac1344f741189efd477f93f75755/users/3413b5c1b8554c418cde751313e73273/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.236788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7028ac1344f741189efd477f93f75755, user_id=3413b5c1b8554c418cde751313e73273, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.239980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.259706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1021/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:29 2022] PUT /identity/v3/projects/7028ac1344f741189efd477f93f75755/users/3413b5c1b8554c418cde751313e73273/roles/87f1831257824560813f954348534540 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:29.272794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.273083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.273360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.307704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] MFA Rules not processed for user `3413b5c1b8554c418cde751313e73273`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:29.309130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.337717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.339912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.378577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:29.400913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:29.418717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d09fe-64ce-4d54-a5a2-32634ced1b5b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.426454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.427003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.427286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.427630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] PATH_INFO: `/v3/projects/b776df921106476c9e95321f8c6c1e59` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.428982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b776df921106476c9e95321f8c6c1e59)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.431290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.449669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1020/2041] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:29.463221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.464720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.501422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b776df921106476c9e95321f8c6c1e59', 'request_id': 'req-758245dd-8b8e-409d-9c63-44192825ecb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.514370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b776df921106476c9e95321f8c6c1e59', 'request_id': 'req-758245dd-8b8e-409d-9c63-44192825ecb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.524834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b776df921106476c9e95321f8c6c1e59', 'request_id': 'req-758245dd-8b8e-409d-9c63-44192825ecb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.537646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d161e-a67d-4625-8582-81bc7e1c410b None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.538463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] The token cache is being invalidate because project b776df921106476c9e95321f8c6c1e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:29.538894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.539600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-758245dd-8b8e-409d-9c63-44192825ecb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:29.541349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1022/2042] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:29 2022] DELETE /identity/v3/projects/b776df921106476c9e95321f8c6c1e59 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:29.545658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d193-b01a-4571-9b61-de1716435dec None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.545941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d193-b01a-4571-9b61-de1716435dec None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.546214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d193-b01a-4571-9b61-de1716435dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.566515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ee48d193-b01a-4571-9b61-de1716435dec None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:29.567736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ee48d193-b01a-4571-9b61-de1716435dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.595040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.595690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.595970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.596291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.598060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.646679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1023/2043] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:29.654791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.655080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.655356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.668873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.670305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.675827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:29.677141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:29.680379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1021/2044] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:29.685653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1024/2045] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:29.690060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.700786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f96424-679e-4d90-af04-f01e7bb1f2cc service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.705462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.706198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.706475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.706745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.707397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:29.707725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.709324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.721305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1022/2046] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:29.725473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.735370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3b644-7d2d-4729-9cad-e27bca4c7a3f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.751792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.751792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.751792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.751792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.752055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.752055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.775151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1025/2047] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:29 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:29.779585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.789791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c6d18-ae60-4fe4-bdc0-21f538feee43 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.816089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:29.852165 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.852816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.854493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.854771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.855409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.857365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.884197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1023/2048] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:29 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:29.885401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.897488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70e98a-88ed-4aec-955e-6981fc147fd2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.903298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.903948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.904232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.904495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.905139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.906753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.917056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1026/2049] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:29.921231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.932919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600d96d-35bf-40ae-af39-d6f227f131ae admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.938272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c57af-851f-4bbf-b29e-410ba5f10cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.938922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c57af-851f-4bbf-b29e-410ba5f10cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.939211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c57af-851f-4bbf-b29e-410ba5f10cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.939622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c57af-851f-4bbf-b29e-410ba5f10cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:29.940282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c57af-851f-4bbf-b29e-410ba5f10cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:29.941882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c57af-851f-4bbf-b29e-410ba5f10cd6 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:29.956723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1024/2050] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:29.964506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:29.977074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082866af-39c7-42e0-bc86-a642027e4de2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:29.982253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:29.983061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:29.983356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:29.983673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] PATH_INFO: `/v3/projects/03fd70f792494a8c928194e9db7d90fa/users/cf3d083691dd4645916ff8445548b116/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:30.003618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fd70f792494a8c928194e9db7d90fa, user_id=cf3d083691dd4645916ff8445548b116, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:30.007230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:30.042592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1027/2051] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:29 2022] PUT /identity/v3/projects/03fd70f792494a8c928194e9db7d90fa/users/cf3d083691dd4645916ff8445548b116/roles/87f1831257824560813f954348534540 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:30.054327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:30.054609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:30.054880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:30.083532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] MFA Rules not processed for user `cf3d083691dd4645916ff8445548b116`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:30.084792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:30.150745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1025/2052] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:30.161950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:30.163634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:30.183841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:30.223498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2aa02-5847-412e-b91c-f583cf712324 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:30.227171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:30.227799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:30.228079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:30.228351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:30.237071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:30.238516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:30.247307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1028/2053] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:38.301952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:38.313876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79068143-7099-43db-a1c1-b9ae954d26a5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:38.321486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:38.322215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:38.322585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:38.323044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:38.325531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:38.396174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:38.397585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9ae54-289e-4856-a2c1-41d5692d03f3 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:38.408817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1026/2054] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:40.243377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=18 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:48.136399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:48.140934 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:48.239260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c88bf0-2b84-44c5-ad2d-66d82cdfddda service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:48.247204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:48.247900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:48.248664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:48.248664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] PATH_INFO: `/v3/users/3413b5c1b8554c418cde751313e73273` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:48.250583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3413b5c1b8554c418cde751313e73273)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:48.254527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:48.325616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3413b5c1b8554c418cde751313e73273'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.342392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3413b5c1b8554c418cde751313e73273'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.343230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3413b5c1b8554c418cde751313e73273'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.344045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3413b5c1b8554c418cde751313e73273'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.370803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3413b5c1b8554c418cde751313e73273'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.402796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:48.403610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1029/2055] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:48 2022] DELETE /identity/v3/users/3413b5c1b8554c418cde751313e73273 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:48.408166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:48.409500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:48.409996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:48.441083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:48.442291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:48.457380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:48.458832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:48.527615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1027/2056] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:48.538277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:48.545843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf959c21-5822-42ca-8712-54076243cbce admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:48.553714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:48.554760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:48.556697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:48.557595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:48.558268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] PATH_INFO: `/v3/users/529ed8ad430545b996e5e227cabff52c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:48.559919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] RBAC: Authorizing `identity:delete_user(user_id=529ed8ad430545b996e5e227cabff52c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:48.571516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:48.600477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:48.632151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529ed8ad430545b996e5e227cabff52c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.633767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcee9d-cdd0-4952-8de3-3525e5b660c1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:48.638081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa18b37-ec11-408b-a796-2697981065eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:48.638748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa18b37-ec11-408b-a796-2697981065eb service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:48.639025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa18b37-ec11-408b-a796-2697981065eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:48.639283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa18b37-ec11-408b-a796-2697981065eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:48.639655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529ed8ad430545b996e5e227cabff52c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.640307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529ed8ad430545b996e5e227cabff52c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.640923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529ed8ad430545b996e5e227cabff52c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.648529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529ed8ad430545b996e5e227cabff52c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:48.652432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa18b37-ec11-408b-a796-2697981065eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:48.653926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa18b37-ec11-408b-a796-2697981065eb service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:48.664275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1028/2057] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:48.667579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1030/2058] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:48 2022] DELETE /identity/v3/users/529ed8ad430545b996e5e227cabff52c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:48.672504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:48.672793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:48.673069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:48.693367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:48.694584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:48.785156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1029/2059] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:48.797125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:48.798596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:48.863554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf037a2-ceb2-4803-9464-b855f71ab91a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:48.867225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:48.868470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:48.868745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:48.869014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:48.878552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:48.880027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:48.895080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1031/2060] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:49.090695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.092419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.105374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366325f-638a-4c34-b42e-199bc5152051 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.110512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.111130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.111406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.111712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] PATH_INFO: `/v3/users/cf3d083691dd4645916ff8445548b116` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.112833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3d083691dd4645916ff8445548b116)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.114744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.155040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.158001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.173784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bcdb8-0d5e-48c8-ade2-188e885ce551 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.179800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.180495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.180772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.181042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] PATH_INFO: `/v3/projects/7028ac1344f741189efd477f93f75755` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.182219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7028ac1344f741189efd477f93f75755)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.184227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.196533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:49.216832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3d083691dd4645916ff8445548b116'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.231013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3d083691dd4645916ff8445548b116'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.231852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3d083691dd4645916ff8445548b116'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.252562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3d083691dd4645916ff8445548b116'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.253553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e1d508b2-e902-4fb9-99d6-2acd216f194e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3d083691dd4645916ff8445548b116'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.286960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1030/2061] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:49 2022] DELETE /identity/v3/users/cf3d083691dd4645916ff8445548b116 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:49.293241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f227039-c635-43fc-878f-f64985529df6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.293528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f227039-c635-43fc-878f-f64985529df6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.293818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f227039-c635-43fc-878f-f64985529df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.297780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7028ac1344f741189efd477f93f75755', 'request_id': 'req-87114e1c-169a-42c6-ad21-1cde75e1186f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.317114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7028ac1344f741189efd477f93f75755', 'request_id': 'req-87114e1c-169a-42c6-ad21-1cde75e1186f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.329955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7028ac1344f741189efd477f93f75755', 'request_id': 'req-87114e1c-169a-42c6-ad21-1cde75e1186f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.346628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] The token cache is being invalidate because project 7028ac1344f741189efd477f93f75755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:49.347152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.349056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.349539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4f227039-c635-43fc-878f-f64985529df6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:49.350885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4f227039-c635-43fc-878f-f64985529df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.352829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1032/2062] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:49 2022] DELETE /identity/v3/projects/7028ac1344f741189efd477f93f75755 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:49.353968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.355461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.412448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:49.443841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1031/2063] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:49.443841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.443841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.443841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.483746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:49.485673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.497956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1032/2064] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:49.498635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.500482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.513111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87114e1c-169a-42c6-ad21-1cde75e1186f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.528647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.529329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.529593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.530008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] PATH_INFO: `/v3/projects/ed48010356f246aabdedc3f2855d2f00` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.536157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed48010356f246aabdedc3f2855d2f00)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.537721 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.596943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5ccdf-170c-49fe-a7e4-f83b00289beb None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.615940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed48010356f246aabdedc3f2855d2f00', 'request_id': 'req-a2e93c7d-ee05-454d-baea-1adc9c11eb53'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.623322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed48010356f246aabdedc3f2855d2f00', 'request_id': 'req-a2e93c7d-ee05-454d-baea-1adc9c11eb53'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.637517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed48010356f246aabdedc3f2855d2f00', 'request_id': 'req-a2e93c7d-ee05-454d-baea-1adc9c11eb53'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.678883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] The token cache is being invalidate because project ed48010356f246aabdedc3f2855d2f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:49.679192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.689062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a2e93c7d-ee05-454d-baea-1adc9c11eb53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.695157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1033/2065] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:49 2022] DELETE /identity/v3/projects/ed48010356f246aabdedc3f2855d2f00 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:49.697009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.697192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.697696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.698007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.698285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.698557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.698801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.704653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.728657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:49.730392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.744563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:49.839469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1034/2066] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:49.840510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.841965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:49.844079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.845880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.857356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f99657-73fd-4d53-ad80-9bcc2dbcf808 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.863889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.864521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.864797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.865066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] PATH_INFO: `/v3/users/36bffc3bf8504742b9e007e7ef7f5ce0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.865554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1033/2067] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:49.866191 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.866364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bffc3bf8504742b9e007e7ef7f5ce0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.868533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.888805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618a03d-07da-4ffd-bef4-ac3081f57a91 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.896108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.896866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.897220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.907615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.908512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:49.908995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.910744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.926741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1034/2068] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:49.931669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:49.943778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937c21a-e266-489b-9725-fdb4edb0f777 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:49.949073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bffc3bf8504742b9e007e7ef7f5ce0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.949819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:49.950685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:49.951039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:49.951450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:49.952210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:49.956024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:49.957341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:49.967243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bffc3bf8504742b9e007e7ef7f5ce0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.968066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bffc3bf8504742b9e007e7ef7f5ce0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.970122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bffc3bf8504742b9e007e7ef7f5ce0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:49.980740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bffc3bf8504742b9e007e7ef7f5ce0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.007616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1035/2069] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:49 2022] DELETE /identity/v3/users/36bffc3bf8504742b9e007e7ef7f5ce0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:50.007821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1035/2070] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:49 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:50.011162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.012647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.105407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd965caa-9491-46d5-b62e-7e680f28d9b4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.177144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.177801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.178472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.178472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.178998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.181287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.212385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1036/2071] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:50.216758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.229020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a614a-e8a6-4825-adb9-7d0987291812 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.257672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.257672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.257672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.257672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.257987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.257987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.282138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1036/2072] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:50.286015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.299062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c355b2-c52b-4f48-8cd0-f572d23366ea admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.319946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b6d7c-dfbc-45fe-8d48-f05fcfb80d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.322320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6d7c-dfbc-45fe-8d48-f05fcfb80d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.323426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6d7c-dfbc-45fe-8d48-f05fcfb80d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.324340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6d7c-dfbc-45fe-8d48-f05fcfb80d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.325407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b6d7c-dfbc-45fe-8d48-f05fcfb80d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.327235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b6d7c-dfbc-45fe-8d48-f05fcfb80d9d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.339525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1037/2073] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:50.343652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.362187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b970a4f-de92-4e2a-88b3-e2f8b1c8e9c4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.375402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.376144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.376422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.376690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] PATH_INFO: `/v3/projects/37167f7abcf44559a071d7a4860e30f7/users/8de1d03f386f43c89b157707855bc8eb/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.394684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37167f7abcf44559a071d7a4860e30f7, user_id=8de1d03f386f43c89b157707855bc8eb, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.398157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.414563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1037/2074] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:50 2022] PUT /identity/v3/projects/37167f7abcf44559a071d7a4860e30f7/users/8de1d03f386f43c89b157707855bc8eb/roles/87f1831257824560813f954348534540 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:50.425914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.426240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.426476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.433112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.436732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.458596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] MFA Rules not processed for user `8de1d03f386f43c89b157707855bc8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:50.460744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.513722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:50.526898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4059e74-a2af-4d16-8041-1b6819bba13e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.534812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.535639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.535924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.536195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] PATH_INFO: `/v3/projects/03fd70f792494a8c928194e9db7d90fa` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.538279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03fd70f792494a8c928194e9db7d90fa)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.540253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.546644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1038/2075] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:50.558642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.560511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.576398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ba588-a86c-45a0-be75-1415f57a37f2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.583518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.584240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.584518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.587831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] PATH_INFO: `/v3/projects/4507ea6a38b24c26ba4ce61c2abc0970` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.589750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4507ea6a38b24c26ba4ce61c2abc0970)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.592238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.620586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:50.622455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03fd70f792494a8c928194e9db7d90fa', 'request_id': 'req-237d2098-280e-4326-886d-db424139bc16'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.639399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03fd70f792494a8c928194e9db7d90fa', 'request_id': 'req-237d2098-280e-4326-886d-db424139bc16'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.655267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03fd70f792494a8c928194e9db7d90fa', 'request_id': 'req-237d2098-280e-4326-886d-db424139bc16'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.672357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4507ea6a38b24c26ba4ce61c2abc0970', 'request_id': 'req-debe6969-6566-4fc6-8bad-ee062bfe937c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.674008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] The token cache is being invalidate because project 03fd70f792494a8c928194e9db7d90fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:50.674293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.675707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.687278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1038/2076] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:50 2022] DELETE /identity/v3/projects/03fd70f792494a8c928194e9db7d90fa => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:50.687278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.687975 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4507ea6a38b24c26ba4ce61c2abc0970', 'request_id': 'req-debe6969-6566-4fc6-8bad-ee062bfe937c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.688060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.703135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4507ea6a38b24c26ba4ce61c2abc0970', 'request_id': 'req-debe6969-6566-4fc6-8bad-ee062bfe937c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.724936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] The token cache is being invalidate because project 4507ea6a38b24c26ba4ce61c2abc0970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:50.725236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.725898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-debe6969-6566-4fc6-8bad-ee062bfe937c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:50.727622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1039/2077] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:50 2022] DELETE /identity/v3/projects/4507ea6a38b24c26ba4ce61c2abc0970 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:50.732029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d70dd-4010-42eb-a997-e099490af3ec None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.732318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d70dd-4010-42eb-a997-e099490af3ec None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.733277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d70dd-4010-42eb-a997-e099490af3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.753105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-912d70dd-4010-42eb-a997-e099490af3ec None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:50.754407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-912d70dd-4010-42eb-a997-e099490af3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.755992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d2098-280e-4326-886d-db424139bc16 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.819334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.819969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.820243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.820511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.822264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.825167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1040/2078] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:50.826412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eba63-2125-4412-8ca6-fe9ab96f518f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.826685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eba63-2125-4412-8ca6-fe9ab96f518f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.827016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eba63-2125-4412-8ca6-fe9ab96f518f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.850933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c23eba63-2125-4412-8ca6-fe9ab96f518f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:50.852190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c23eba63-2125-4412-8ca6-fe9ab96f518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.860085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1041/2079] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:50.861260 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23592579-7328-4135-95b2-5666b10e2835 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.861541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23592579-7328-4135-95b2-5666b10e2835 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.861809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23592579-7328-4135-95b2-5666b10e2835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.881249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-23592579-7328-4135-95b2-5666b10e2835 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:50.882333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-23592579-7328-4135-95b2-5666b10e2835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.885929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.887349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.890680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1042/2080] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:50.891687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.891968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.892241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.899568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1039/2081] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:50.900372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.911760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476545b1-74c2-4f9a-bdaf-b131b8d97d97 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.913019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:50.914209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:50.922301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1043/2082] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:50.923973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.924655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.924934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.925207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.926120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:50.926345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.926510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.928467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.936147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787050-09a8-4773-b7d7-0bc8dae906d7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.940920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1040/2083] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:50.941047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.941456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.941733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.942002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.942676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:50.942981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.944668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.945315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.957350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d690-0bde-458a-a49d-acf81836209e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.960555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1044/2084] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:45:50.961395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:50.962284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.962913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.963140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.963555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.964186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.966236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.971877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb15b4-5fe8-4d5a-b5dd-602d17f43449 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:50.980793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:50.981398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:50.981674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:50.981944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:50.982574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:50.984540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:50.994671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1041/2085] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:50.998757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.013029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:51.019850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1045/2086] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:51.025435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.041980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:51.074997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1e5fb-af6f-41b7-834b-9a2a0d031703 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.102360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c00e-a383-46fe-a3ea-6dd68d78d7e4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.108657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.109391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.109664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.109930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.110559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.112531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.137917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1046/2087] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:51 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:51.142251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.143398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.144135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.144398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.144880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.145391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.147355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.154249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979022e-8fd9-4416-a875-72d4352668c3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.159951 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.160925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.161466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.161966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.162968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.164835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.171912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1042/2088] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:50 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:51.175928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.176069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1047/2089] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:51.180677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.187562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66bbeb-7680-4103-a9a5-f1bd50c5e719 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.191766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0db4af-2ad3-4bd3-abcb-d99f2acee8a3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.192643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.193306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.193676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.194463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.194636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.196268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.197642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.197785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.197954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.198231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.199014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.201141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.208868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1043/2090] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:51.213130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.214007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1048/2091] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:51.218153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.227255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5c925-9efe-4a29-b172-4068f90b4160 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.228027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4deef3d-89e1-4e8e-b6b1-d5640f8c5227 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.232721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.233143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570c012-1ab2-45c2-a38e-b26a87137cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.233599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.233943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c012-1ab2-45c2-a38e-b26a87137cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.234047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.234459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c012-1ab2-45c2-a38e-b26a87137cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.234622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.234719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c012-1ab2-45c2-a38e-b26a87137cab admin admin] PATH_INFO: `/v3/projects/9d440a9e2b8448b78bdba4c2df604e3c/users/19eff2834de1432cac6ae8e2983f6ff6/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.235295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.237189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.250003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1044/2092] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:51.253355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570c012-1ab2-45c2-a38e-b26a87137cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d440a9e2b8448b78bdba4c2df604e3c, user_id=19eff2834de1432cac6ae8e2983f6ff6, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.256597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570c012-1ab2-45c2-a38e-b26a87137cab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.267366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.279549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1049/2093] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:51 2022] PUT /identity/v3/projects/9d440a9e2b8448b78bdba4c2df604e3c/users/19eff2834de1432cac6ae8e2983f6ff6/roles/87f1831257824560813f954348534540 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:51.285901 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264e12e-368e-4ffa-891e-bddfe2f830dc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.288275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.288658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.289131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.352465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] MFA Rules not processed for user `19eff2834de1432cac6ae8e2983f6ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:51.355783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:51.371231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2cbfe-5c1b-4952-bfe1-9f024c85b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.372397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2cbfe-5c1b-4952-bfe1-9f024c85b0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.374738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2cbfe-5c1b-4952-bfe1-9f024c85b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.374738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2cbfe-5c1b-4952-bfe1-9f024c85b0d1 admin admin] PATH_INFO: `/v3/projects/3f3eae8c6bb040f28f2f5612dd7328f5/users/8d5e1f2fc4ea429f8200b4bc35dab9da/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.392442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2cbfe-5c1b-4952-bfe1-9f024c85b0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3eae8c6bb040f28f2f5612dd7328f5, user_id=8d5e1f2fc4ea429f8200b4bc35dab9da, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.396008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2cbfe-5c1b-4952-bfe1-9f024c85b0d1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.412054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1045/2094] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:51 2022] PUT /identity/v3/projects/3f3eae8c6bb040f28f2f5612dd7328f5/users/8d5e1f2fc4ea429f8200b4bc35dab9da/roles/87f1831257824560813f954348534540 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:51.424216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.424495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.424772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.444403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:51.455220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] MFA Rules not processed for user `8d5e1f2fc4ea429f8200b4bc35dab9da`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:51.456612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:51.457540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1050/2095] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:51.462163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.473186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e718d3c-477d-4af4-83b7-5448b0f8aead None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.519012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:51.545056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1046/2096] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:51.545883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.546666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.546985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.547703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.547945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.549997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.557454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.558869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:51.579509 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1051/2097] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:51 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:51.582314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.619044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bfed4-6ab1-4580-bfa9-6d5028df6ee5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.647286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e572-4764-47e6-a5c1-36f8c0e58ed3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.652209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.652804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.653083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.653352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.653981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.656504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.674571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.674571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.674571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.674571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.683664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1052/2098] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:51 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:51.687179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.697351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4c26-1581-4df5-b987-850514fa4fc1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.702947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.703569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.703846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.704413 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.704917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.706494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.719470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1053/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:51.723693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.742025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfaaa4-1334-40c9-8bc1-38501dadd613 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.751077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.751600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.751778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.752059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.752332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] PATH_INFO: `/v3/projects/a8befef8f9f1444c8c2e1ed443ee7f9a/users/4fb42418f9f54e10a5716f86531cd43a/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.753352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.762755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1047/2100] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:45:51.770002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8befef8f9f1444c8c2e1ed443ee7f9a, user_id=4fb42418f9f54e10a5716f86531cd43a, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.772968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.790873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1054/2101] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:51 2022] PUT /identity/v3/projects/a8befef8f9f1444c8c2e1ed443ee7f9a/users/4fb42418f9f54e10a5716f86531cd43a/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:51.794889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.806333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5b94-a579-42ea-9854-b8fddda8fbc8 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.812460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:51.869692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5685b6-292d-4c27-aabe-f27249da41f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.870504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5685b6-292d-4c27-aabe-f27249da41f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.870848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5685b6-292d-4c27-aabe-f27249da41f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.871198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5685b6-292d-4c27-aabe-f27249da41f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.871936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5685b6-292d-4c27-aabe-f27249da41f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.873592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5685b6-292d-4c27-aabe-f27249da41f9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.885889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1048/2102] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:51.890473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:51.902067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235c2f5-1efb-4a94-8c05-cb9b53edf523 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:51.907763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:51.908590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.908901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.909200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] PATH_INFO: `/v3/projects/a8befef8f9f1444c8c2e1ed443ee7f9a/users/4fb42418f9f54e10a5716f86531cd43a/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.911365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8befef8f9f1444c8c2e1ed443ee7f9a, user_id=4fb42418f9f54e10a5716f86531cd43a, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:51.914067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:51.930676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1055/2103] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:51 2022] PUT /identity/v3/projects/a8befef8f9f1444c8c2e1ed443ee7f9a/users/4fb42418f9f54e10a5716f86531cd43a/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:51.943021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:51.943550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:51.943966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:51.988055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] MFA Rules not processed for user `4fb42418f9f54e10a5716f86531cd43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:51.989314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:52.070449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1049/2104] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:52.084889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:52.143649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5a86-83f3-454f-a407-615826dd1a5b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:52.147285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacd95c-59f2-47ec-9860-0e750887c062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:52.147909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacd95c-59f2-47ec-9860-0e750887c062 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:52.148189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacd95c-59f2-47ec-9860-0e750887c062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:52.148464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacd95c-59f2-47ec-9860-0e750887c062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:52.157379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacd95c-59f2-47ec-9860-0e750887c062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:52.158746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacd95c-59f2-47ec-9860-0e750887c062 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:52.168775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1056/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:52.548014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:52.615665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e92ad-6111-4cf0-8bbe-df789f6c5428 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:52.620119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:52.620743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:52.621019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:52.621287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] PATH_INFO: `/v3/users/19eff2834de1432cac6ae8e2983f6ff6` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:52.622400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19eff2834de1432cac6ae8e2983f6ff6)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:52.624692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:52.665338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:52.680452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19eff2834de1432cac6ae8e2983f6ff6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:52.688469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19eff2834de1432cac6ae8e2983f6ff6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:52.689056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19eff2834de1432cac6ae8e2983f6ff6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:52.696898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19eff2834de1432cac6ae8e2983f6ff6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:52.697224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19eff2834de1432cac6ae8e2983f6ff6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:52.714416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1050/2106] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:52 2022] DELETE /identity/v3/users/19eff2834de1432cac6ae8e2983f6ff6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:52.719443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b478-7141-47ff-9281-808d2504c586 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:52.719940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b478-7141-47ff-9281-808d2504c586 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:52.720051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b478-7141-47ff-9281-808d2504c586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:52.739087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c837b478-7141-47ff-9281-808d2504c586 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:52.740569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c837b478-7141-47ff-9281-808d2504c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:52.749205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:52.806111 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1057/2107] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:52.815762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:52.817160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:52.879023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3605-1ca8-4e55-bb95-bc9b4c1d4b1a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:52.883107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:52.883818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:52.884167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:52.884510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:52.894378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:52.895840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:52.906072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1051/2108] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:53.245183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b478-7141-47ff-9281-808d2504c586 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:53.246644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c837b478-7141-47ff-9281-808d2504c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:53.260783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b478-7141-47ff-9281-808d2504c586 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:53.266861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:53.267602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:53.267997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:53.268387 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] PATH_INFO: `/v3/projects/9d440a9e2b8448b78bdba4c2df604e3c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:53.269709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d440a9e2b8448b78bdba4c2df604e3c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:53.272193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:53.318800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d440a9e2b8448b78bdba4c2df604e3c', 'request_id': 'req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.324441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d440a9e2b8448b78bdba4c2df604e3c', 'request_id': 'req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.332171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d440a9e2b8448b78bdba4c2df604e3c', 'request_id': 'req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.342877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] The token cache is being invalidate because project 9d440a9e2b8448b78bdba4c2df604e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:53.343165 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.343786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.345243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1058/2109] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:53 2022] DELETE /identity/v3/projects/9d440a9e2b8448b78bdba4c2df604e3c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:53.349609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:53.359093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc73c91-c74e-40ff-b5e1-29f2ef942edc service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:53.413862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:53.414485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:53.414762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:53.415034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] PATH_INFO: `/v3/users/4fb42418f9f54e10a5716f86531cd43a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:53.416169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb42418f9f54e10a5716f86531cd43a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:53.418067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:53.467158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb42418f9f54e10a5716f86531cd43a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.473943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb42418f9f54e10a5716f86531cd43a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.474761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb42418f9f54e10a5716f86531cd43a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.475990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:53.482736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb42418f9f54e10a5716f86531cd43a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.483850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-67283c6b-c562-4eea-b519-5d605097926f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb42418f9f54e10a5716f86531cd43a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.501486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1052/2110] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:53 2022] DELETE /identity/v3/users/4fb42418f9f54e10a5716f86531cd43a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:53.771009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:53.772970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:53.794183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d57e0-4221-4b4d-8f55-1ef297ab31c5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:53.860910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:53.861668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:53.861953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:53.862224 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] PATH_INFO: `/v3/projects/a8befef8f9f1444c8c2e1ed443ee7f9a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:53.863463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8befef8f9f1444c8c2e1ed443ee7f9a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:53.865409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:53.909172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:53.917205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8befef8f9f1444c8c2e1ed443ee7f9a', 'request_id': 'req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.924875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8befef8f9f1444c8c2e1ed443ee7f9a', 'request_id': 'req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.932806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8befef8f9f1444c8c2e1ed443ee7f9a', 'request_id': 'req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.944691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] The token cache is being invalidate because project a8befef8f9f1444c8c2e1ed443ee7f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:53.944980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.945622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3d4ec54a-406f-4c8d-a0c3-8f22ef4aeb23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:53.947101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1059/2111] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:53 2022] DELETE /identity/v3/projects/a8befef8f9f1444c8c2e1ed443ee7f9a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:53.952065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:53.952511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:53.952595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:53.972539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:53.973763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:54.036115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1053/2112] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:54.043631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.043917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.044193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.064097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:54.065292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:54.073367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1060/2113] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:54.077400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.086856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426cff3-c85b-4341-8a6a-9793aaa49a50 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.091312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.091922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.092195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.092460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.093637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:54.093937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.095530 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.106723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1054/2114] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:54.111298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.120458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df091a73-ab90-45b2-af8c-7e1584dfa1d2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.125068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.125647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.125948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.126213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.126827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.128767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.159381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1061/2115] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:54.160707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.171523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fce3ea-a089-4584-b47b-a0cee365dec7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.230813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.231411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.231717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.232010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.232632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.235103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.257382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1055/2116] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:54.262012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.272052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942565b-aa5b-4f54-8d99-b1dabdf14fab admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.277110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.277713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.277994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.278266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.278917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.281167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.291461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1062/2117] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:54.295473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.306193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a36b2-ed8b-4352-b457-fb70975176b9 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.311396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b5e18-bb86-4830-b84b-45ff4d3c00eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.312016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5e18-bb86-4830-b84b-45ff4d3c00eb admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.312292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5e18-bb86-4830-b84b-45ff4d3c00eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.312559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5e18-bb86-4830-b84b-45ff4d3c00eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.313227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b5e18-bb86-4830-b84b-45ff4d3c00eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.314804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b5e18-bb86-4830-b84b-45ff4d3c00eb admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.325546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1056/2118] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:54.329826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.339824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4c72-d753-4c24-a922-ebbffd7b5b2b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.346004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.346826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.347145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.347370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] PATH_INFO: `/v3/projects/c91f46903922434a9141b793a7c287da/users/2c204c91d9924ec084f0cc43bac39909/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.363375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91f46903922434a9141b793a7c287da, user_id=2c204c91d9924ec084f0cc43bac39909, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.366072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.380206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1063/2119] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:54 2022] PUT /identity/v3/projects/c91f46903922434a9141b793a7c287da/users/2c204c91d9924ec084f0cc43bac39909/roles/87f1831257824560813f954348534540 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:54.391416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.391731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.392009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.419251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] MFA Rules not processed for user `2c204c91d9924ec084f0cc43bac39909`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:54.420468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:54.445785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:54.491330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1057/2120] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:54.495667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.557220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe6900-4f31-4173-9e67-89cc18ba2486 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.561823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.562412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.562688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.562962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.563621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.565506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.595130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1064/2121] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:54.599065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.609824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e89890-7eb7-4aae-beef-dd226bdaffd4 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.706144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.706763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.707040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.707314 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.707994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.709917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.743218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1058/2122] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:54.745594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.756841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f4566-7dfd-41f3-aa0d-fa9ae80972b4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.761709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.762307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.762583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.762858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.763510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.765807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.776608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1065/2123] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:54.780802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.790783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7453c-b4a0-4201-bc4f-8c5ee2ea4fc0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.795415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.796075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.796366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.796620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] PATH_INFO: `/v3/projects/8d675d216189417d9c7376a45820cc96/users/2f8a52dbd93c47ae8141ad98a5032da3/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.812186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d675d216189417d9c7376a45820cc96, user_id=2f8a52dbd93c47ae8141ad98a5032da3, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.814859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.829237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1059/2124] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:54 2022] PUT /identity/v3/projects/8d675d216189417d9c7376a45820cc96/users/2f8a52dbd93c47ae8141ad98a5032da3/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:54.833069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.844489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2982e7b-7f13-45de-a950-068d7ef70070 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.883919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:54.912570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11744cb1-9cd9-4c36-ac9c-ceb7baa58c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.913169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11744cb1-9cd9-4c36-ac9c-ceb7baa58c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.913445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11744cb1-9cd9-4c36-ac9c-ceb7baa58c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.913707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11744cb1-9cd9-4c36-ac9c-ceb7baa58c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.914435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11744cb1-9cd9-4c36-ac9c-ceb7baa58c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.916013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11744cb1-9cd9-4c36-ac9c-ceb7baa58c6f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.926835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1066/2125] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:45:54.931282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:54.942830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ae77-186b-49f9-a330-42a7cac998a4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:54.947940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:54.948603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.948976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.949306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] PATH_INFO: `/v3/projects/8d675d216189417d9c7376a45820cc96/users/2f8a52dbd93c47ae8141ad98a5032da3/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:54.951160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d675d216189417d9c7376a45820cc96, user_id=2f8a52dbd93c47ae8141ad98a5032da3, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:54.954460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:54.967862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1060/2126] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:54 2022] PUT /identity/v3/projects/8d675d216189417d9c7376a45820cc96/users/2f8a52dbd93c47ae8141ad98a5032da3/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:54.978995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:54.979279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:54.979582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:55.008535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] MFA Rules not processed for user `2f8a52dbd93c47ae8141ad98a5032da3`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:55.009950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:55.093500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1067/2127] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:55.106411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:55.109022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:55.182256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d1d7-ee2d-4ff6-a5b1-43edc927f217 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:55.198894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:55.199626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:55.199907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:55.200172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:55.210992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:55.212538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:55.225069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1061/2128] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:55.573298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:55.584339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c56c97-77d6-4ae8-a41d-bddb436b6166 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:55.588999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8a52dbd93c47ae8141ad98a5032da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d675d216189417d9c7376a45820cc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:55.590301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:55.590801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:55.591174 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:55.591911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:55.592435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:55.594638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:55.623531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1068/2129] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:55.624068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:55.634167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce791fe1-2bb6-4e97-8ecb-361d44548894 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:55.638720 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8a52dbd93c47ae8141ad98a5032da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d675d216189417d9c7376a45820cc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:55.639316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:55.639665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:55.639961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:55.640588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:55.642477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:55.663558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:55.674660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1062/2130] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:55 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:45:55.676066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:55.686207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e5ec-b53a-4aa0-8bd4-cdc9cecfa47e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:55.741555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8a52dbd93c47ae8141ad98a5032da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d675d216189417d9c7376a45820cc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:55.742158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:55.742442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:55.742717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:55.743358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:55.743699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:55.745306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:55.758009 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1069/2131] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:55.794597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:55.805472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9e9fa-e8ae-4cae-8cee-85250e08ac4b tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:55.810048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f8a52dbd93c47ae8141ad98a5032da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d675d216189417d9c7376a45820cc96', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:55.810657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:55.810939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:55.811218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] PATH_INFO: `/v3/projects/36922f7fcdd04981ad20642d0e29416b` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:55.812429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorizing `identity:delete_project(project_id=36922f7fcdd04981ad20642d0e29416b)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:55.814306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:55.866762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36922f7fcdd04981ad20642d0e29416b', 'request_id': 'req-8713aed4-f888-497d-ad8d-0daff54c813e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:55.872619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36922f7fcdd04981ad20642d0e29416b', 'request_id': 'req-8713aed4-f888-497d-ad8d-0daff54c813e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:55.882237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36922f7fcdd04981ad20642d0e29416b', 'request_id': 'req-8713aed4-f888-497d-ad8d-0daff54c813e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:55.895050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] The token cache is being invalidate because project 36922f7fcdd04981ad20642d0e29416b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:55.895345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:55.896160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8713aed4-f888-497d-ad8d-0daff54c813e tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:55.897972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1063/2132] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:55 2022] DELETE /identity/v3/projects/36922f7fcdd04981ad20642d0e29416b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:56.094777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:56.096686 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:56.141857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:56.169203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259ee1c-5a87-4ee0-9617-fef5716404ce tempest-VolumeQuotaClassesTest-1285678938 tempest-VolumeQuotaClassesTest-1285678938-project-admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:56.254668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:56.255835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:56.256417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:56.256754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] PATH_INFO: `/v3/users/2c204c91d9924ec084f0cc43bac39909` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:56.259155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c204c91d9924ec084f0cc43bac39909)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:56.260595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:56.336319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c204c91d9924ec084f0cc43bac39909'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:56.346193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c204c91d9924ec084f0cc43bac39909'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:56.346942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c204c91d9924ec084f0cc43bac39909'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:56.347723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c204c91d9924ec084f0cc43bac39909'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:56.359203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c204c91d9924ec084f0cc43bac39909'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:56.381472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1070/2133] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:56 2022] DELETE /identity/v3/users/2c204c91d9924ec084f0cc43bac39909 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:56.385648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:56.386176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:56.386176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:56.407058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:56.416018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:56.479247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1064/2134] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:56.490269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:56.491919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:56.554527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623329ca-bbee-4622-a36c-4f762750fb75 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:56.558225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:56.558821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:56.559096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:56.559364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:56.569956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:56.571403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:56.586285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1071/2135] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:57.032567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:57.034089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:57.051239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1746ab-6a93-4cbe-b91c-c807fd8495f9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:57.058758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:57.059517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:57.059849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:57.060074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] PATH_INFO: `/v3/projects/c91f46903922434a9141b793a7c287da` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:57.061247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91f46903922434a9141b793a7c287da)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:57.063520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:57.133820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91f46903922434a9141b793a7c287da', 'request_id': 'req-86a56efc-f22e-4550-9301-3858648fda7d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.139856 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91f46903922434a9141b793a7c287da', 'request_id': 'req-86a56efc-f22e-4550-9301-3858648fda7d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.149626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91f46903922434a9141b793a7c287da', 'request_id': 'req-86a56efc-f22e-4550-9301-3858648fda7d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.162077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] The token cache is being invalidate because project c91f46903922434a9141b793a7c287da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:57.162421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.163090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.164807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1065/2136] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:57 2022] DELETE /identity/v3/projects/c91f46903922434a9141b793a7c287da => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:57.168799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:57.183979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e025f-6f30-4222-a2c5-002d6c9ac4fb service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:57.188665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:57.247930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:57.248556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:57.248897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:57.249166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] PATH_INFO: `/v3/users/2f8a52dbd93c47ae8141ad98a5032da3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:57.250281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8a52dbd93c47ae8141ad98a5032da3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:57.252233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:57.297593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8a52dbd93c47ae8141ad98a5032da3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.303109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8a52dbd93c47ae8141ad98a5032da3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.303756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8a52dbd93c47ae8141ad98a5032da3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.304360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8a52dbd93c47ae8141ad98a5032da3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.311365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-336197ad-ec7e-4c08-956c-21f94f72dff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8a52dbd93c47ae8141ad98a5032da3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.328541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1072/2137] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:45:57 2022] DELETE /identity/v3/users/2f8a52dbd93c47ae8141ad98a5032da3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:57.841350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:57.843362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:57.857105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a56efc-f22e-4550-9301-3858648fda7d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:57.861404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:57.927534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:57.928112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:57.928386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:57.928654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] PATH_INFO: `/v3/projects/8d675d216189417d9c7376a45820cc96` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:57.929771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d675d216189417d9c7376a45820cc96)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:57.931688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:57.989633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d675d216189417d9c7376a45820cc96', 'request_id': 'req-cec4a826-4905-45d4-9779-d117321b3605'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:57.996029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d675d216189417d9c7376a45820cc96', 'request_id': 'req-cec4a826-4905-45d4-9779-d117321b3605'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:58.003999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d675d216189417d9c7376a45820cc96', 'request_id': 'req-cec4a826-4905-45d4-9779-d117321b3605'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:58.016565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] The token cache is being invalidate because project 8d675d216189417d9c7376a45820cc96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:45:58.016980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:58.017634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cec4a826-4905-45d4-9779-d117321b3605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:45:58.019134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1066/2138] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:45:57 2022] DELETE /identity/v3/projects/8d675d216189417d9c7376a45820cc96 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:58.024110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.024389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.024659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.043672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:58.044858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:58.112169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1073/2139] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:45:58.119919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.120205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.120484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.142359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:58.143760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:58.152400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1067/2140] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:58.156678 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.167033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4476ab7-18f0-42dd-ba4d-5201233726f9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.173978 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.175530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.175871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.176415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.176875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:45:58.177255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.178912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.202287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1074/2141] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:45:58.204564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.215507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f470-d589-486e-9ce6-0a9e909f054d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.221609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.222215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.222487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.222757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.223390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.225614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.267538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1068/2142] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:58.272061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.284806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca55b47-a83c-4d0a-96ff-a151e84b7a4c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.316845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:58.437858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.439282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.439778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.440157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.441042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.443109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.469604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1075/2143] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:58.470847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.482862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545c8c-7db9-4e7e-8509-f373fe824377 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.488262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.488944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.489449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.489738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.490358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.492133 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.503265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1069/2144] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:58.508420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.519261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6399b-fc05-4510-b40f-f031615c9471 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.524212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286d6ee-358e-4bb2-9378-d3bdf897a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.524885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d6ee-358e-4bb2-9378-d3bdf897a836 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.525090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d6ee-358e-4bb2-9378-d3bdf897a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.525335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d6ee-358e-4bb2-9378-d3bdf897a836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.526002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286d6ee-358e-4bb2-9378-d3bdf897a836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.527588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286d6ee-358e-4bb2-9378-d3bdf897a836 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.541591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1076/2145] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:58.546830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.556992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe143c80-bdd1-4032-8704-43ff20881801 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.562714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.563365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.563790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.564218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] PATH_INFO: `/v3/projects/d544c9bb4c054ea98b994f2a63e73515/users/59c23df24bde4be1ba5d5bd417f227c4/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.580584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d544c9bb4c054ea98b994f2a63e73515, user_id=59c23df24bde4be1ba5d5bd417f227c4, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.583345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.600523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1070/2146] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:58 2022] PUT /identity/v3/projects/d544c9bb4c054ea98b994f2a63e73515/users/59c23df24bde4be1ba5d5bd417f227c4/roles/87f1831257824560813f954348534540 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:45:58.615708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.616060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.616426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.650931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] MFA Rules not processed for user `59c23df24bde4be1ba5d5bd417f227c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:58.652284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:58.731535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1077/2147] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:58.736442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.773982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:58.806097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dac9da-2598-4895-8566-590b8dc83d14 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.811320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.811950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.812230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.812503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.813135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.815023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.840510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1071/2148] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:58.845657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.856469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602792a-d8cc-4a0d-a2ec-75e9bd3d3b00 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.920653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.921257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.921535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.921806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.922443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.924415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.947695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1078/2149] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:45:58 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:45:58.948758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.959341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ffc77-15a4-45c8-b866-64555e241b06 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.963891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.964471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.964747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.965012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:58.965641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:58.967215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:58.978263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1072/2150] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:58.982180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:58.991969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fafa6d-0f6d-458b-859f-f849ce36c782 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:58.996829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:58.997453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:58.997729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:58.998002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] PATH_INFO: `/v3/projects/ed519bb8868d492c92d3f4c56cecda7c/users/50615167a7154db49c1297ce047eb344/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:59.010348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:59.015227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed519bb8868d492c92d3f4c56cecda7c, user_id=50615167a7154db49c1297ce047eb344, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:59.018656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:59.034230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1079/2151] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:58 2022] PUT /identity/v3/projects/ed519bb8868d492c92d3f4c56cecda7c/users/50615167a7154db49c1297ce047eb344/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:59.039290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:59.049779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d35049-02a4-4f52-bc6a-82001a5b6c0e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:59.109546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca22793-6fbe-47ce-b4ef-594085ac1739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:59.110159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca22793-6fbe-47ce-b4ef-594085ac1739 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:59.110438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca22793-6fbe-47ce-b4ef-594085ac1739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:59.110714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca22793-6fbe-47ce-b4ef-594085ac1739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:59.111352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca22793-6fbe-47ce-b4ef-594085ac1739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:59.112972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca22793-6fbe-47ce-b4ef-594085ac1739 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:59.123034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1073/2152] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:45:59.127350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:59.137797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f41c7-d633-461a-b06b-fde51ad5a2f2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:59.142901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:59.143611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:59.143895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:59.144166 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] PATH_INFO: `/v3/projects/ed519bb8868d492c92d3f4c56cecda7c/users/50615167a7154db49c1297ce047eb344/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:59.146225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed519bb8868d492c92d3f4c56cecda7c, user_id=50615167a7154db49c1297ce047eb344, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:59.148900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:59.159905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1080/2153] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:45:59 2022] PUT /identity/v3/projects/ed519bb8868d492c92d3f4c56cecda7c/users/50615167a7154db49c1297ce047eb344/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:45:59.169871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:59.170158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:59.170430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:59.198339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] MFA Rules not processed for user `50615167a7154db49c1297ce047eb344`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:45:59.199468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:59.269740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1074/2154] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:45:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:45:59.281628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:59.283379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:59.340231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f195e8-f80e-4186-8d53-3b4fae2ce9d2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:59.343851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:59.344435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:59.344709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:59.344973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:59.353570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:59.354985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:59.364475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1081/2155] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:45:59.445146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:45:59.446598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:45:59.459346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cae5e-e55d-43b8-8aa9-56c1e16c41a8 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:45:59.463013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30bc18-721b-4a49-ad79-922b5728d207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:45:59.463656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30bc18-721b-4a49-ad79-922b5728d207 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:45:59.463939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30bc18-721b-4a49-ad79-922b5728d207 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:45:59.464220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30bc18-721b-4a49-ad79-922b5728d207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:45:59.490771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:45:59.531515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30bc18-721b-4a49-ad79-922b5728d207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:45:59.533011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30bc18-721b-4a49-ad79-922b5728d207 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:45:59.542300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1075/2156] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:45:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:09.361968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=35,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:09.538159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=15 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:24.663925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:24.732679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb074b5-0bf6-4385-81e9-bce65565f0be service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:24.737201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:24.737823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:24.738102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:24.738370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] PATH_INFO: `/v3/users/59c23df24bde4be1ba5d5bd417f227c4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:24.739539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c23df24bde4be1ba5d5bd417f227c4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:24.748308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:24.813005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c23df24bde4be1ba5d5bd417f227c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:24.821772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c23df24bde4be1ba5d5bd417f227c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:24.822418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c23df24bde4be1ba5d5bd417f227c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:24.823022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c23df24bde4be1ba5d5bd417f227c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:24.833461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c23df24bde4be1ba5d5bd417f227c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:24.864720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1082/2157] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:24 2022] DELETE /identity/v3/users/59c23df24bde4be1ba5d5bd417f227c4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:24.869361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:24.869899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:24.869899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:24.899806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:24.901025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:24.975203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1076/2158] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:24.985050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:24.986466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:25.066487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f7234-ac5c-45fc-ac69-1e445cf73b89 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:25.071087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:25.071958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:25.072406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:25.072777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:25.103627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:25.110615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:25.136765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1083/2159] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:25.463157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:25.464713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:25.487052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348d2e2-3daa-4fbc-b211-c8e2075503a6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:25.492096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:25.492812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:25.493172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:25.493516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] PATH_INFO: `/v3/projects/d544c9bb4c054ea98b994f2a63e73515` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:25.495533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d544c9bb4c054ea98b994f2a63e73515)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:25.497369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:25.585371 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd544c9bb4c054ea98b994f2a63e73515', 'request_id': 'req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.597182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd544c9bb4c054ea98b994f2a63e73515', 'request_id': 'req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.630737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd544c9bb4c054ea98b994f2a63e73515', 'request_id': 'req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.649772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] The token cache is being invalidate because project d544c9bb4c054ea98b994f2a63e73515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:25.650078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.652813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.655459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1077/2160] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:25 2022] DELETE /identity/v3/projects/d544c9bb4c054ea98b994f2a63e73515 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:25.660873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:25.675641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d9168-f5b3-4bca-8856-d524df49d204 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:25.700362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:25.768276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:25.769274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:25.775974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:25.776347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] PATH_INFO: `/v3/users/50615167a7154db49c1297ce047eb344` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:25.777737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50615167a7154db49c1297ce047eb344)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:25.780271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:25.878842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50615167a7154db49c1297ce047eb344'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.890954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50615167a7154db49c1297ce047eb344'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.892006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50615167a7154db49c1297ce047eb344'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.893692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50615167a7154db49c1297ce047eb344'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.909271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57e2a64-184f-4b4d-a037-a9b41999db51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50615167a7154db49c1297ce047eb344'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:25.930756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1084/2161] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:25 2022] DELETE /identity/v3/users/50615167a7154db49c1297ce047eb344 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:26.421375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:26.422362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:26.436907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65c4b5-e7b8-412a-a7b7-de90c2cb4197 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:26.492179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:26.492803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.493081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.493351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] PATH_INFO: `/v3/projects/ed519bb8868d492c92d3f4c56cecda7c` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.494482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed519bb8868d492c92d3f4c56cecda7c)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:26.496420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:26.527658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:26.544162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed519bb8868d492c92d3f4c56cecda7c', 'request_id': 'req-1edb2fc0-e5a6-499b-817a-de1e077f8efe'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:26.549551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed519bb8868d492c92d3f4c56cecda7c', 'request_id': 'req-1edb2fc0-e5a6-499b-817a-de1e077f8efe'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:26.556265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed519bb8868d492c92d3f4c56cecda7c', 'request_id': 'req-1edb2fc0-e5a6-499b-817a-de1e077f8efe'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:26.567381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] The token cache is being invalidate because project ed519bb8868d492c92d3f4c56cecda7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:26.567705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:26.568329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1edb2fc0-e5a6-499b-817a-de1e077f8efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:26.569777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1078/2162] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:26 2022] DELETE /identity/v3/projects/ed519bb8868d492c92d3f4c56cecda7c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:26.574564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.574850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.575150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.594823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:26.596416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:26.655612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1085/2163] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:46:26.663534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.663821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.664098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.684808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:26.686665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:26.694477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1079/2164] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:26.698711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:26.708327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acaa31-a706-4c90-9d6a-5af019bba1d5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:26.712811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:26.713397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.713671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.713937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.714573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:26.714869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:26.716488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:26.727716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1086/2165] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:26.731633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:26.741140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfb605-084e-4c06-8ff3-56473e030f13 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:26.745971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:26.746556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.746830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.747091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.748362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:26.750225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:26.778700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1080/2166] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:26 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:26.782511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:26.802659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f8746-70bf-4b11-908e-cd58939fbe55 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:26.832006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:26.883364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:26.884189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.884624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.885032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.885871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:26.888373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:26.913608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1087/2167] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:26 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:26.915659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:26.926296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c957-779d-40cd-a96d-2d8021c025f1 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:26.930868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:26.931516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.931753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.932017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.932657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:26.934215 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:26.944812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1081/2168] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:26.949040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:26.962450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2038d-ba22-4df7-8bf3-d4849a2d2b5c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:26.970657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96ee2a-7bad-4999-847e-441235098b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:26.971407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96ee2a-7bad-4999-847e-441235098b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:26.971840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96ee2a-7bad-4999-847e-441235098b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:26.972263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96ee2a-7bad-4999-847e-441235098b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:26.975829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96ee2a-7bad-4999-847e-441235098b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:26.975829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96ee2a-7bad-4999-847e-441235098b0b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:26.988900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1088/2169] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:26.993101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:27.004170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61feb967-257b-4342-8521-5c188acffa92 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:27.009509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:27.010288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.010662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.011026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] PATH_INFO: `/v3/projects/0171d5a9af074c3cac9aba3e56cd2751/users/ab2c8ad1657d4f739071a875e962d4ad/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.027158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0171d5a9af074c3cac9aba3e56cd2751, user_id=ab2c8ad1657d4f739071a875e962d4ad, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:27.030508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:27.047821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1082/2170] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:26 2022] PUT /identity/v3/projects/0171d5a9af074c3cac9aba3e56cd2751/users/ab2c8ad1657d4f739071a875e962d4ad/roles/87f1831257824560813f954348534540 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:27.060375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.060375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.060666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.090132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] MFA Rules not processed for user `ab2c8ad1657d4f739071a875e962d4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:27.092117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:27.183723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1089/2171] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:27.194910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:27.196725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:27.257676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24210-a39a-46e4-a690-00b8aeb272dd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:27.261333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78cda-46cc-48c8-8c25-918cb2413f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:27.261958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78cda-46cc-48c8-8c25-918cb2413f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.262236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78cda-46cc-48c8-8c25-918cb2413f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.262506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78cda-46cc-48c8-8c25-918cb2413f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.270592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78cda-46cc-48c8-8c25-918cb2413f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:27.272054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78cda-46cc-48c8-8c25-918cb2413f5d service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:27.281166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1083/2172] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:27.300134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:27.368908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b236cf-fedb-4773-ae50-c7f29132864f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:27.373487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:27.374119 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.374399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.374667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] PATH_INFO: `/v3/users/ab2c8ad1657d4f739071a875e962d4ad` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.375854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2c8ad1657d4f739071a875e962d4ad)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:27.377763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:27.418645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:27.427459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2c8ad1657d4f739071a875e962d4ad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:27.435798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2c8ad1657d4f739071a875e962d4ad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:27.436496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2c8ad1657d4f739071a875e962d4ad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:27.437334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2c8ad1657d4f739071a875e962d4ad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:27.445962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2c8ad1657d4f739071a875e962d4ad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:27.464312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1090/2173] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:27 2022] DELETE /identity/v3/users/ab2c8ad1657d4f739071a875e962d4ad => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:27.469378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.469829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.469920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.488516 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:27.489710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:27.537282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:27.551640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1084/2174] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:27.561318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:27.562727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:27.619668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3737fb-924f-4a01-a896-34eb9204334a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:27.623982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3f47a-db95-4731-a5b7-3c906ff12c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:27.624579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3f47a-db95-4731-a5b7-3c906ff12c9e service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.624855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3f47a-db95-4731-a5b7-3c906ff12c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.625123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3f47a-db95-4731-a5b7-3c906ff12c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.634334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3f47a-db95-4731-a5b7-3c906ff12c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:27.635800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3f47a-db95-4731-a5b7-3c906ff12c9e service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:27.645604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1091/2175] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:27.916492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:27.918459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:27.932658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef5f26-e6b4-4927-a077-ff6525b56b48 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:27.938584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:27.939333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:27.939685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:27.940024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] PATH_INFO: `/v3/projects/0171d5a9af074c3cac9aba3e56cd2751` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:27.941582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0171d5a9af074c3cac9aba3e56cd2751)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:27.943707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:27.992061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0171d5a9af074c3cac9aba3e56cd2751', 'request_id': 'req-43114407-eb7c-43b9-a6b4-2d248c0cf822'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:27.997577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0171d5a9af074c3cac9aba3e56cd2751', 'request_id': 'req-43114407-eb7c-43b9-a6b4-2d248c0cf822'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:28.005362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0171d5a9af074c3cac9aba3e56cd2751', 'request_id': 'req-43114407-eb7c-43b9-a6b4-2d248c0cf822'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:28.017788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] The token cache is being invalidate because project 0171d5a9af074c3cac9aba3e56cd2751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:28.018078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:28.018749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43114407-eb7c-43b9-a6b4-2d248c0cf822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:28.020281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1085/2176] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:27 2022] DELETE /identity/v3/projects/0171d5a9af074c3cac9aba3e56cd2751 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:28.025089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.025561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.025648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.045926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:28.047041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:28.103954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1092/2177] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:28.112195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.112502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.112776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.132468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:28.133651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:28.141809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1086/2178] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:28.145825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.155863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3643902-45e7-4a41-ad01-433b7a022b08 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.160364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.161046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.162525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.162980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.163700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:28.164003 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.165620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.181449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1093/2179] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:28.184942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.195189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3328-9381-4732-8cac-2afae19a30c7 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.202179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.202781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.203326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.203533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.204501 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.206688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.238954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1087/2180] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:28.240148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.249944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f91b4-bb03-4dbb-b90c-2cb653cdb82d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.300451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:28.317195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.317928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.318273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.318778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.319760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.321656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.347005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1094/2181] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:46:28.351156 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.361961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779375c-4940-45d5-bf67-a728047d302c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.366335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.366935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.367209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.367507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.368160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.369747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.380535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1088/2182] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:28.384640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.395205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10925ed6-b7ce-40a5-ac71-f22b1e4d55aa admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.399925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e580afc-6926-405e-926f-75cebff9e029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.400516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580afc-6926-405e-926f-75cebff9e029 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.400792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580afc-6926-405e-926f-75cebff9e029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.401065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580afc-6926-405e-926f-75cebff9e029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.401709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e580afc-6926-405e-926f-75cebff9e029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.403296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e580afc-6926-405e-926f-75cebff9e029 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.414405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1095/2183] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:28.418837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.428968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ccdf-9958-4aed-b369-24f9feb088ae admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.433609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.434325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.434674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.435061 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] PATH_INFO: `/v3/projects/fdf0425874504d439c5f344fccfc66b9/users/19e4851edd0546f083995f935c81b998/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.451953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf0425874504d439c5f344fccfc66b9, user_id=19e4851edd0546f083995f935c81b998, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.454684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.467990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1089/2184] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:28 2022] PUT /identity/v3/projects/fdf0425874504d439c5f344fccfc66b9/users/19e4851edd0546f083995f935c81b998/roles/87f1831257824560813f954348534540 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:28.478055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.478337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.478605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.506171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] MFA Rules not processed for user `19e4851edd0546f083995f935c81b998`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:28.507596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:28.570530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1096/2185] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:28.574885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.637826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d634b0-6d16-42d1-b4a7-a868d40b2895 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.643031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.643667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.644073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.644348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.644978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.646862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.667542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:28.681343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1090/2186] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:28.685700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.696649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f2cc9-0d8d-4316-87a1-b2b892e77a53 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.756096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.756765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.757022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.757295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.757931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.759926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.781174 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1097/2187] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:28.785043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.795179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012f868-b023-41c3-8325-438795092d0c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.799921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.800503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.800781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.801054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.801698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.803291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.815411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1091/2188] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:28.819360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.829802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9002f-eb1d-47d4-96c9-4ba6a421ce02 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.834264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.834888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.835162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.835430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] PATH_INFO: `/v3/projects/17969478d33a405f898808dbcb641c9a/users/16f094cd81a84cc3a105c979c50c521b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.849854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=17969478d33a405f898808dbcb641c9a, user_id=16f094cd81a84cc3a105c979c50c521b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.852527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.865691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1098/2189] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:28 2022] PUT /identity/v3/projects/17969478d33a405f898808dbcb641c9a/users/16f094cd81a84cc3a105c979c50c521b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:28.869618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.880253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693d824-743b-4dad-b9f8-5b3bd03c24dc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.936321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e4c00-c8fe-4c45-9aa0-9b76cb965197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.936989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e4c00-c8fe-4c45-9aa0-9b76cb965197 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.937187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e4c00-c8fe-4c45-9aa0-9b76cb965197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.937641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e4c00-c8fe-4c45-9aa0-9b76cb965197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.938288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e4c00-c8fe-4c45-9aa0-9b76cb965197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.939930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e4c00-c8fe-4c45-9aa0-9b76cb965197 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.949682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1092/2190] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:28.953745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:28.963980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbd9ca-0577-40c8-925c-9d938ba463ab admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:28.970188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:28.971186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:28.971720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:28.972132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] PATH_INFO: `/v3/projects/17969478d33a405f898808dbcb641c9a/users/16f094cd81a84cc3a105c979c50c521b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:28.974704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17969478d33a405f898808dbcb641c9a, user_id=16f094cd81a84cc3a105c979c50c521b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:28.977468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:28.988570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1099/2191] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:28 2022] PUT /identity/v3/projects/17969478d33a405f898808dbcb641c9a/users/16f094cd81a84cc3a105c979c50c521b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:29.002207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:29.002207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:29.002207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:29.030418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] MFA Rules not processed for user `16f094cd81a84cc3a105c979c50c521b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:29.030418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:29.099972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1093/2192] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:29.111010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:29.112788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:29.123008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:29.170506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6db31-005e-4cf3-9cfd-de010332734d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:29.174102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1317f6-e141-43e2-8ada-5458eb5d29c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:29.174680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1317f6-e141-43e2-8ada-5458eb5d29c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:29.174953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1317f6-e141-43e2-8ada-5458eb5d29c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:29.175220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1317f6-e141-43e2-8ada-5458eb5d29c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:29.230495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1317f6-e141-43e2-8ada-5458eb5d29c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:29.232924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1317f6-e141-43e2-8ada-5458eb5d29c6 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:29.242738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1100/2193] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:32.558103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:32.559717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:32.623161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52e10-3495-4125-857a-615fbe8aa576 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:32.627648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:32.628264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:32.628538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:32.628802 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] PATH_INFO: `/v3/users/8de1d03f386f43c89b157707855bc8eb` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:32.629896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de1d03f386f43c89b157707855bc8eb)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:32.632068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:32.642742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:32.678694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de1d03f386f43c89b157707855bc8eb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:32.687549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de1d03f386f43c89b157707855bc8eb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:32.688163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de1d03f386f43c89b157707855bc8eb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:32.696957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de1d03f386f43c89b157707855bc8eb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:32.697748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de1d03f386f43c89b157707855bc8eb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:32.715927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1094/2194] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:32 2022] DELETE /identity/v3/users/8de1d03f386f43c89b157707855bc8eb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:32.720892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:32.721341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:32.721428 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:32.741184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:32.742625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:32.814687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1101/2195] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:46:32.826409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:32.827858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:32.891023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed55e-2fc0-4ca7-8b46-57428a36805d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:32.894826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:32.895453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:32.895756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:32.896036 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:32.920557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:32.922109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:32.932468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1095/2196] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:33.241906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.243473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.255914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fab4a-d357-4010-aa8e-09cb244d4efc None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.260542 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.261169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.261454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.261727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] PATH_INFO: `/v3/users/34dd63dc1c314f2e8fd5a3ae9b6e9c67` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.262845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=34dd63dc1c314f2e8fd5a3ae9b6e9c67)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.264823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.310486 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.312702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.321273 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34dd63dc1c314f2e8fd5a3ae9b6e9c67'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.330821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34dd63dc1c314f2e8fd5a3ae9b6e9c67'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.332947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34dd63dc1c314f2e8fd5a3ae9b6e9c67'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.334143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34dd63dc1c314f2e8fd5a3ae9b6e9c67'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.336413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a69a-30be-472b-a7ce-16d626af9f53 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.337236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:33.343782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.344603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.344995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.345103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] PATH_INFO: `/v3/projects/37167f7abcf44559a071d7a4860e30f7` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.346628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-988b6aab-3629-4576-bbec-61034b80b7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34dd63dc1c314f2e8fd5a3ae9b6e9c67'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.347077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] RBAC: Authorizing `identity:delete_project(project_id=37167f7abcf44559a071d7a4860e30f7)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.349811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.379647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1102/2197] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:33 2022] DELETE /identity/v3/users/34dd63dc1c314f2e8fd5a3ae9b6e9c67 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:33.385499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.385960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.386339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.409975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37167f7abcf44559a071d7a4860e30f7', 'request_id': 'req-edfb028e-411e-4fba-b90d-e0a77bbe775d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.416217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:33.417616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37167f7abcf44559a071d7a4860e30f7', 'request_id': 'req-edfb028e-411e-4fba-b90d-e0a77bbe775d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.418050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.431159 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37167f7abcf44559a071d7a4860e30f7', 'request_id': 'req-edfb028e-411e-4fba-b90d-e0a77bbe775d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.446604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] The token cache is being invalidate because project 37167f7abcf44559a071d7a4860e30f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:33.446902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.447686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-edfb028e-411e-4fba-b90d-e0a77bbe775d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:33.453793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1096/2198] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:33 2022] DELETE /identity/v3/projects/37167f7abcf44559a071d7a4860e30f7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:33.455335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148272f-2dfd-4bea-8cec-760c43bcf9d8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.455666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148272f-2dfd-4bea-8cec-760c43bcf9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.456019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148272f-2dfd-4bea-8cec-760c43bcf9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.477167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b148272f-2dfd-4bea-8cec-760c43bcf9d8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:33.478588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b148272f-2dfd-4bea-8cec-760c43bcf9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.490542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:33.524324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1103/2199] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:33.533081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.534767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.560623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1097/2200] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:33.567659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.568055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.568414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.591581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:33.592921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.602020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1098/2201] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:46:33.606695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.615596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcbec0-ac2d-412e-a05c-e5e3cfb9ee65 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.616580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b9d3c-747f-4d12-a88a-efdf63974fb6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.620427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.621020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.621295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.621439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.621548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.622209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:33.622496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.624071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.624180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.624432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.624731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.635553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.636887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1099/2202] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:46:33.641016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.642028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.647866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1104/2203] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:46:33.653349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bdce3-fa3c-428c-8529-aad29a104069 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.658787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.659417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.659740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.660085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.660654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.662553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.701737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1100/2204] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:33 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:46:33.706020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.717008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4405-fba2-4259-9883-d66ebc69f587 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.779685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.780303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.780559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.780833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.781471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.783392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.820434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1105/2205] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:33 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:33.824510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.835210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c695382-4179-4d54-ad9f-12a6021f977d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.842245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.843136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.843687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.844604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.846203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.848291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.864636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1101/2206] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:33.868891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.881038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101052-aa33-4366-bf48-162d9d5e0ae2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.885909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.886355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.886638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.886900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.888329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.890014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.891425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:33.905862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1106/2207] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:33.908303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.920918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce248a9-fe22-4038-bcdd-dfab57403534 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.928303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:33.928973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:33.929255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:33.929528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] PATH_INFO: `/v3/projects/39a872cd1b1e451ba5786fe246950f73/users/b8e4bda3331540d1a0b869270db9c25d/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:33.951333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a872cd1b1e451ba5786fe246950f73, user_id=b8e4bda3331540d1a0b869270db9c25d, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:33.952471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.954348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:33.954465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:33.970194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1102/2208] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:33 2022] PUT /identity/v3/projects/39a872cd1b1e451ba5786fe246950f73/users/b8e4bda3331540d1a0b869270db9c25d/roles/87f1831257824560813f954348534540 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:33.970757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633a8d-5cd8-4358-8f5e-0b23a3613aee admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:33.971399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:33.974944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.040187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:34.040830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.041108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.041377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] PATH_INFO: `/v3/users/19e4851edd0546f083995f935c81b998` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.042500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e4851edd0546f083995f935c81b998)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:34.044511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:34.049524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca4b3c-d9da-4ced-a014-fd7a762879fc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:34.054990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:34.055624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.056025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.056401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] PATH_INFO: `/v3/projects/a91ae1f7a6b54543927220c058e184a7` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.057446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a91ae1f7a6b54543927220c058e184a7)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:34.060020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:34.101496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e4851edd0546f083995f935c81b998'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.110777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:34.115258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e4851edd0546f083995f935c81b998'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.116767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e4851edd0546f083995f935c81b998'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.117570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e4851edd0546f083995f935c81b998'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.126784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1ad56a7f-bf59-44d8-8ef9-c0160868ad84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e4851edd0546f083995f935c81b998'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.128105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a91ae1f7a6b54543927220c058e184a7', 'request_id': 'req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.136688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a91ae1f7a6b54543927220c058e184a7', 'request_id': 'req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.147761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a91ae1f7a6b54543927220c058e184a7', 'request_id': 'req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.153350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1107/2209] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:33 2022] DELETE /identity/v3/users/19e4851edd0546f083995f935c81b998 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:34.154386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.154664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.154982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.160828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] The token cache is being invalidate because project a91ae1f7a6b54543927220c058e184a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:34.161242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.162668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3aa9c04d-b20d-41b2-9f21-68329afa0ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.164488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1103/2210] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:33 2022] DELETE /identity/v3/projects/a91ae1f7a6b54543927220c058e184a7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:34.165678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.166055 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.166424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.178658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:34.184191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.187947 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:34.189372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.251979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1108/2211] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:34.252884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:34.257121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.271277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462823d-ec91-437e-9d1a-cba0c68d89f4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:34.276689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:34.278252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1104/2212] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:34.278389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.279300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:34.280837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.282844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.282844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] PATH_INFO: `/v3/users/815042d616d04bb7b10245defcecd620` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.283307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815042d616d04bb7b10245defcecd620)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:34.285419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:34.300169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:34.355621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864863-dcac-4272-89af-0c95ff05f214 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:34.359239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815042d616d04bb7b10245defcecd620'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.360536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:34.361345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.361657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.361927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.370026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815042d616d04bb7b10245defcecd620'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.371251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815042d616d04bb7b10245defcecd620'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.372839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815042d616d04bb7b10245defcecd620'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.375306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:34.377175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:34.381657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815042d616d04bb7b10245defcecd620'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:34.388272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1105/2213] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:34.397005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:34.407426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1109/2214] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:34 2022] DELETE /identity/v3/users/815042d616d04bb7b10245defcecd620 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:34.411776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb2dc7-70da-48f4-9be7-81d4476569aa service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:34.479722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:34.480572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.481248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.481248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.484819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.523095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:34.579513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:34.581380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df937e-f3bf-4b72-8557-c453d090e580 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:34.594440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1106/2215] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:34.888197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:34.889716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:34.939981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17926b-5f84-4aec-b1e7-6aa2e5ed8eb3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:34.960324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:34.961215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:34.961750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:34.962066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] PATH_INFO: `/v3/projects/4529b1d98bb94ee7a0ce691312b2599e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:34.963653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4529b1d98bb94ee7a0ce691312b2599e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:34.965682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.039255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4529b1d98bb94ee7a0ce691312b2599e', 'request_id': 'req-cb965162-666d-45fa-96c2-186949b3bd6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:35.046405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4529b1d98bb94ee7a0ce691312b2599e', 'request_id': 'req-cb965162-666d-45fa-96c2-186949b3bd6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:35.054319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4529b1d98bb94ee7a0ce691312b2599e', 'request_id': 'req-cb965162-666d-45fa-96c2-186949b3bd6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:35.066402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] The token cache is being invalidate because project 4529b1d98bb94ee7a0ce691312b2599e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:35.066824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:35.067503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cb965162-666d-45fa-96c2-186949b3bd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:35.069235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1110/2216] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:34 2022] DELETE /identity/v3/projects/4529b1d98bb94ee7a0ce691312b2599e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:35.073912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.074200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.074474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.096731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:35.097948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:35.169256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1107/2217] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:35.177887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.178188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.178485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.202396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:35.203675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:35.214530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1111/2218] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:35.219248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.232725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8861b6-9dde-4fed-8510-40dd8e049e45 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.238358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.239038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.239262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.239566 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.240229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:35.240537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.260242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.312255 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1108/2219] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:35.316944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.330049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd394bf-fa30-486c-831f-615b274d548b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.336582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.337410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.337694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.338043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.339552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.342347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.389264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1112/2220] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:35 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:35.399094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.410533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754abfe-edc6-4d5f-96da-02394fcc74b2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.489579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.490259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.490611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.490950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.491712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.493700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.519517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1109/2221] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:35 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:35.522033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.533447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762521-337b-41be-ae1c-cf0f6025e2e0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.538368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.539000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.539279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.539603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.540250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.541846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.556352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1113/2222] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:35.559251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.570690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e205916-c5ad-4a8b-ad86-728ec3912dd0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.576442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5b73a-9283-45a0-9aee-776ac803ba2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.581728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5b73a-9283-45a0-9aee-776ac803ba2b admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.582325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5b73a-9283-45a0-9aee-776ac803ba2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.582435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5b73a-9283-45a0-9aee-776ac803ba2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.583311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5b73a-9283-45a0-9aee-776ac803ba2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.585601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5b73a-9283-45a0-9aee-776ac803ba2b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.600223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1110/2223] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:35.604512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.628930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c031b-040d-4c19-9379-4774c477b609 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.636141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.636803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.637110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.637391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] PATH_INFO: `/v3/projects/705ce7e25b8b4d3e9b60b20f9e6a9c06/users/d30e9f0cee6a4fc2b5700ff5860098aa/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.673723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=705ce7e25b8b4d3e9b60b20f9e6a9c06, user_id=d30e9f0cee6a4fc2b5700ff5860098aa, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.673723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.692409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1114/2224] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:35 2022] PUT /identity/v3/projects/705ce7e25b8b4d3e9b60b20f9e6a9c06/users/d30e9f0cee6a4fc2b5700ff5860098aa/roles/87f1831257824560813f954348534540 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:35.697259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42041d2f-c721-4aaf-8752-458694fe54f3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.697547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42041d2f-c721-4aaf-8752-458694fe54f3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.698589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42041d2f-c721-4aaf-8752-458694fe54f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.771287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-42041d2f-c721-4aaf-8752-458694fe54f3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:35.775803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-42041d2f-c721-4aaf-8752-458694fe54f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:35.828776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:35.836226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:35.838380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:35.855319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=17,keystone:SELECT=79,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:35.900676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1111/2225] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:35.953881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.954160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.954432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.968085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75051e7-6baf-4714-ba9a-df52033c27cf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:35.984888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:35.985551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:35.985941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:35.986120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] PATH_INFO: `/v3/projects/fdf0425874504d439c5f344fccfc66b9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:35.987257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf0425874504d439c5f344fccfc66b9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:35.989216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:35.990858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] MFA Rules not processed for user `b8e4bda3331540d1a0b869270db9c25d`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:35.992323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.080437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf0425874504d439c5f344fccfc66b9', 'request_id': 'req-8fe09015-d2c3-4ea6-8a17-11147a150a81'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.087799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf0425874504d439c5f344fccfc66b9', 'request_id': 'req-8fe09015-d2c3-4ea6-8a17-11147a150a81'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.099837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf0425874504d439c5f344fccfc66b9', 'request_id': 'req-8fe09015-d2c3-4ea6-8a17-11147a150a81'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.121257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] The token cache is being invalidate because project fdf0425874504d439c5f344fccfc66b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:36.122423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.123518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.126342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1115/2226] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:35 2022] DELETE /identity/v3/projects/fdf0425874504d439c5f344fccfc66b9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:36.130639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:36.150661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.168944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1112/2227] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:46:36.194309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:36.195762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.275019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe09015-d2c3-4ea6-8a17-11147a150a81 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:36.282719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:36.283519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.283913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.284286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] PATH_INFO: `/v3/users/16f094cd81a84cc3a105c979c50c521b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.285965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f094cd81a84cc3a105c979c50c521b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:36.288998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:36.306034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb86f90-2548-44e3-a2ca-c1844e79e5e1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:36.362338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:36.377691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f094cd81a84cc3a105c979c50c521b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.395756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f094cd81a84cc3a105c979c50c521b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.396449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f094cd81a84cc3a105c979c50c521b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.397118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f094cd81a84cc3a105c979c50c521b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.405800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f094cd81a84cc3a105c979c50c521b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.409688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:36.410348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.410625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.410902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.412925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.430161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:36.432234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:36.432748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1116/2228] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:36 2022] DELETE /identity/v3/users/16f094cd81a84cc3a105c979c50c521b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:36.436926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:36.438478 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.445436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1113/2229] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:46:36.446297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:36.447666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.463293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203ea38-8eb4-42bf-b04a-20aa87a4cba6 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:36.476844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:36.504292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e937ed-f644-409c-98e1-ab26609d085d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:36.509646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:36.510905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.511095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.511461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.514730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.534432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:36.535338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.535452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.535885 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] PATH_INFO: `/v3/users/8d5e1f2fc4ea429f8200b4bc35dab9da` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.537272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5e1f2fc4ea429f8200b4bc35dab9da)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:36.539531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:36.593017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:36.594550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce239e8e-f8a3-4871-a1c5-a50c78e42284 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:36.597340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5e1f2fc4ea429f8200b4bc35dab9da'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.605735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1117/2230] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:36.607924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5e1f2fc4ea429f8200b4bc35dab9da'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.609213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5e1f2fc4ea429f8200b4bc35dab9da'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.617328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5e1f2fc4ea429f8200b4bc35dab9da'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.618238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5e1f2fc4ea429f8200b4bc35dab9da'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:36.640152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1114/2231] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:36 2022] DELETE /identity/v3/users/8d5e1f2fc4ea429f8200b4bc35dab9da => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:36.644770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.645444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.645444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.665952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:36.667198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.759358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1118/2232] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:46:36.774109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:36.776671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.895518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a67c22-ef0d-47d0-a511-c058bf8f886f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:36.900349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621d11-c7a2-43cb-8471-c9b7179d921e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:36.901533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621d11-c7a2-43cb-8471-c9b7179d921e service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.901925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621d11-c7a2-43cb-8471-c9b7179d921e service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.902032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:36.902409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621d11-c7a2-43cb-8471-c9b7179d921e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.904306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:36.917016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621d11-c7a2-43cb-8471-c9b7179d921e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:36.919837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621d11-c7a2-43cb-8471-c9b7179d921e service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:36.933872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd23f6-0104-4410-85a6-953554b1d1e3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:36.935892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1115/2233] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:36.940278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:36.940920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:36.941268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:36.941707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] PATH_INFO: `/v3/projects/17969478d33a405f898808dbcb641c9a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:36.942947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17969478d33a405f898808dbcb641c9a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:36.945105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:36.952929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:37.032391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17969478d33a405f898808dbcb641c9a', 'request_id': 'req-f85951a1-d393-47a6-91f5-e770c1dc01c5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.047249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17969478d33a405f898808dbcb641c9a', 'request_id': 'req-f85951a1-d393-47a6-91f5-e770c1dc01c5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.064493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17969478d33a405f898808dbcb641c9a', 'request_id': 'req-f85951a1-d393-47a6-91f5-e770c1dc01c5'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.092541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] The token cache is being invalidate because project 17969478d33a405f898808dbcb641c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:37.093725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.102729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f85951a1-d393-47a6-91f5-e770c1dc01c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.104742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1119/2234] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:36 2022] DELETE /identity/v3/projects/17969478d33a405f898808dbcb641c9a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:37.110790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.111606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.112495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.161531 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:37.163570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:37.256649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:37.307729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1116/2235] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:37.319217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.319631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.319928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.350334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:37.351799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:37.367671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1120/2236] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:37.368179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.395211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e09e9-71d9-4f55-a4f8-2ed9dc3cb534 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.408969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.409846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.410193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.411224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.412349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:37.412727 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.416153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.431239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1117/2237] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:37.435734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.445630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838c514-eaa5-4651-b354-b90a9595b954 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.451312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.451963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.452243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.452517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.453164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.455148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.490784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1121/2238] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:37.492060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.503219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0c7f-da77-4e19-b57b-b3c3b39dc864 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.600758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88973a-6b0a-4858-b8dd-d8572d0cc125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.601918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88973a-6b0a-4858-b8dd-d8572d0cc125 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.602339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88973a-6b0a-4858-b8dd-d8572d0cc125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.602614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88973a-6b0a-4858-b8dd-d8572d0cc125 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.603271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88973a-6b0a-4858-b8dd-d8572d0cc125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.606291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88973a-6b0a-4858-b8dd-d8572d0cc125 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.637006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1118/2239] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:37.643223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.660204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27dbf-6c06-4ee6-bed8-97de20e2ed61 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.660483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27dbf-6c06-4ee6-bed8-97de20e2ed61 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.660752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27dbf-6c06-4ee6-bed8-97de20e2ed61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.662384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4a176-761a-48b4-9f83-5755f7cc0cc1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.669047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.669904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.670253 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.670598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.671330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.673945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.692923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1122/2240] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:37.695068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.701226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-32e27dbf-6c06-4ee6-bed8-97de20e2ed61 None None] MFA Rules not processed for user `d30e9f0cee6a4fc2b5700ff5860098aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:37.702574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-32e27dbf-6c06-4ee6-bed8-97de20e2ed61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:37.708363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f58e4-cf54-42cf-a808-520334cfdf00 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.715308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.716180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.716570 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.716944 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.718380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.720708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.736103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1123/2241] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:37.737150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.749278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8526-1073-4951-9c52-3ece38506d5d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.755168 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.756056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.756420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.756824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] PATH_INFO: `/v3/projects/43d65129e3bf46a68d154912738cf99f/users/36bfc3d59b9a4d68b9fb8a71b521ec02/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.781379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d65129e3bf46a68d154912738cf99f, user_id=36bfc3d59b9a4d68b9fb8a71b521ec02, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.785960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.809236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1124/2242] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:37 2022] PUT /identity/v3/projects/43d65129e3bf46a68d154912738cf99f/users/36bfc3d59b9a4d68b9fb8a71b521ec02/roles/87f1831257824560813f954348534540 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:37.829351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.832045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:37.847976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1119/2243] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:37.847976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.847976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.847976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.882049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] MFA Rules not processed for user `36bfc3d59b9a4d68b9fb8a71b521ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:37.883382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:37.906228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21075355-7c95-4e02-862e-54e6e3baa70c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.910903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:37.911548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:37.911830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:37.912101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] PATH_INFO: `/v3/projects/3f3eae8c6bb040f28f2f5612dd7328f5` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:37.913607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3eae8c6bb040f28f2f5612dd7328f5)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:37.915793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:37.917223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:37.940636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:37.956322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1120/2244] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:37.957148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:37.958424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:37.970196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3eae8c6bb040f28f2f5612dd7328f5', 'request_id': 'req-946ff760-cf56-415e-9afb-383628245ac8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.972067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b8b8-32a3-4c7f-bce1-bc805f66272d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:37.976321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3eae8c6bb040f28f2f5612dd7328f5', 'request_id': 'req-946ff760-cf56-415e-9afb-383628245ac8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.986068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3eae8c6bb040f28f2f5612dd7328f5', 'request_id': 'req-946ff760-cf56-415e-9afb-383628245ac8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.997913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] The token cache is being invalidate because project 3f3eae8c6bb040f28f2f5612dd7328f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:37.998226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:37.998860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:38.000380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1125/2245] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:37 2022] DELETE /identity/v3/projects/3f3eae8c6bb040f28f2f5612dd7328f5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:38.001136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.002397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:38.031403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fa6c2-1580-44b9-bd19-c9254052a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.032033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fa6c2-1580-44b9-bd19-c9254052a2af admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.032307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fa6c2-1580-44b9-bd19-c9254052a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.032578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fa6c2-1580-44b9-bd19-c9254052a2af admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.033199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fa6c2-1580-44b9-bd19-c9254052a2af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.035079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fa6c2-1580-44b9-bd19-c9254052a2af admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.062237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1121/2246] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:37 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:38.063704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.064310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.064590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.084425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:38.084614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ff760-cf56-415e-9afb-383628245ac8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.085543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:38.137515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.138135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.138398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.138670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.140915 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:38.157897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1122/2247] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:38.158715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.166089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:38.182918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081e8b4-9f55-4afd-9bca-d4e26b20c03f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.190651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e4ede-7540-4d31-8434-ab09e1f8f8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.191539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e4ede-7540-4d31-8434-ab09e1f8f8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.191731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e4ede-7540-4d31-8434-ab09e1f8f8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.192065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e4ede-7540-4d31-8434-ab09e1f8f8f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.192767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e4ede-7540-4d31-8434-ab09e1f8f8f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.194831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e4ede-7540-4d31-8434-ab09e1f8f8f7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.204898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.206333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf82f8-013f-4f7c-b004-8636a02f46a5 service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.211634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:38.218221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1126/2248] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:38.219399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.220321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.220709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.235686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1123/2249] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:38 2022] POST /identity/v3/users => generated 293 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:38.235686 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.235960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.236600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.252132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:38.253616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:38.263428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] MFA Rules not processed for user `b8e4bda3331540d1a0b869270db9c25d`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:38.265029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:38.266599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1127/2250] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:38.268679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.283739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005cfef-34fe-4d03-971a-6b3f8bb73629 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.288610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.289727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.290191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.290380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.291248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.293041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.307444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1128/2251] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:38.308237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:38.309642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.321744 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19ea4e-fa4e-44c3-8d43-061af48e09e6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.332876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1124/2252] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:38.334062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.336361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.337831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.338336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.338897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.340332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:38.341182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.342793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.362227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca601b-7518-497e-b1f6-1126b228dbf6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.367525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1129/2253] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:38.371852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.379748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.380701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.380978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.381249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] PATH_INFO: `/v3/projects/378daba287774c368fa20ce00f89e4df/users/91bbba7d956d4390b66aaded767c1370/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.407417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415593-58e9-4e6d-9e62-331cd0420236 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.409033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=378daba287774c368fa20ce00f89e4df, user_id=91bbba7d956d4390b66aaded767c1370, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.411697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.412604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.413548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.414245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.414769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.416000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.418237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.432329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1125/2254] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:38 2022] PUT /identity/v3/projects/378daba287774c368fa20ce00f89e4df/users/91bbba7d956d4390b66aaded767c1370/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:38.436359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.465221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1130/2255] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:38 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:46:38.470749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.525833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28c7c2-10a9-4d5a-97c3-edfe96c867fe admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.546198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b196-e090-4f02-a34c-18dc998570ca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.554986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.555878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.556279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.556609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.557750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.559751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.618356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1131/2256] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:38 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:38.619620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.652812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c2fd7-e99c-4037-b71d-eb1c51dd14ab admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.663784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.664545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.664831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.665098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.665961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.667638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.676019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.676895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.678475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.678852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.680529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.682402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.707307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1132/2257] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:38.711037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1126/2258] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:38.714312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.716272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.727200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93edc5-c713-4299-862d-02f1178af4a9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.740902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f67ee-e774-44dc-b0c4-7febfcc05568 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.745993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eebfa-f2c1-4a33-80b0-9cf2eaa91f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.746807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eebfa-f2c1-4a33-80b0-9cf2eaa91f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.747559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eebfa-f2c1-4a33-80b0-9cf2eaa91f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.747559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eebfa-f2c1-4a33-80b0-9cf2eaa91f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.749060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.749879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.750261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0eebfa-f2c1-4a33-80b0-9cf2eaa91f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.750460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.751023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] PATH_INFO: `/v3/projects/378daba287774c368fa20ce00f89e4df/users/91bbba7d956d4390b66aaded767c1370/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.753699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0eebfa-f2c1-4a33-80b0-9cf2eaa91f7f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.754202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=378daba287774c368fa20ce00f89e4df, user_id=91bbba7d956d4390b66aaded767c1370, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.758209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:38.771927 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1127/2259] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:38 2022] PUT /identity/v3/projects/378daba287774c368fa20ce00f89e4df/users/91bbba7d956d4390b66aaded767c1370/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:38.781429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1133/2260] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:38.782873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:38.905106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:38.919177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76abc9-8d66-4051-96dc-7a3c2ea4d9ea admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:38.939444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:38.940644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:38.941106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:38.941617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] PATH_INFO: `/v3/projects/22831dd6c00c490fb1afd86273c4b829/users/9d7db0c4b18c48b4966fa4cea76c06e0/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:38.962923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22831dd6c00c490fb1afd86273c4b829, user_id=9d7db0c4b18c48b4966fa4cea76c06e0, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:38.966589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:39.004237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1128/2261] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:38 2022] PUT /identity/v3/projects/22831dd6c00c490fb1afd86273c4b829/users/9d7db0c4b18c48b4966fa4cea76c06e0/roles/87f1831257824560813f954348534540 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:39.017681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.018432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.018823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.069245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] MFA Rules not processed for user `9d7db0c4b18c48b4966fa4cea76c06e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:39.070801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:39.218306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1134/2262] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:39.223294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:39.433759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203a268-0e2d-4aa7-bb2d-c87279a33c05 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:39.448141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:39.448829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.449123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.449425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.450329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:39.452447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:39.547933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1129/2263] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:39 2022] POST /identity/v3/projects => generated 385 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:46:39.552275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:39.562712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:39.571029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f74b5-b45f-434d-8941-9280966e2cb6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:39.675207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:39.676022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.676416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.676943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.677747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:39.680474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:39.735120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1135/2264] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:39 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:39.739808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:39.769758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b125dc-8707-4b4f-ba30-a3d77b318fda admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:39.775469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:39.776135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.776535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.776834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.777732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:39.780186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:39.796848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1130/2265] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:39.798660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:39.813911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2139663-3e63-4f6c-95be-625ee142403f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:39.821476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e32eca-ab3c-4210-8ace-5234cde235ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:39.822235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e32eca-ab3c-4210-8ace-5234cde235ec admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.822734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e32eca-ab3c-4210-8ace-5234cde235ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.823048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e32eca-ab3c-4210-8ace-5234cde235ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.823870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e32eca-ab3c-4210-8ace-5234cde235ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:39.825793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e32eca-ab3c-4210-8ace-5234cde235ec admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:39.837130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1136/2266] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:39.841844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:39.857459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a992b-8e90-44e9-a7e9-727f7fc3c7fc admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:39.863440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:39.864232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.864618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.864990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] PATH_INFO: `/v3/projects/d0d4bed87a9545fb97166aee9b3b0489/users/481a4f2768934406b8ad1b710385357c/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.889931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d4bed87a9545fb97166aee9b3b0489, user_id=481a4f2768934406b8ad1b710385357c, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:39.893239 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:39.908850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1131/2267] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:39 2022] PUT /identity/v3/projects/d0d4bed87a9545fb97166aee9b3b0489/users/481a4f2768934406b8ad1b710385357c/roles/87f1831257824560813f954348534540 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:39.919284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:39.919717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:39.920127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:39.980863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] MFA Rules not processed for user `481a4f2768934406b8ad1b710385357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:39.982122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:40.059981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1137/2268] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:40.064533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.139980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b617-f7e5-4da0-88e9-87d67294c52f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.144707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.145307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.145610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.145880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.146519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.148499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.179670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1132/2269] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:40 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:46:40.180843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.192724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345846f1-2bd6-4b1b-abac-b9000573fed5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.257147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.258445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.258445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.258445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.259228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.261307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.297379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1138/2270] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:40 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:46:40.301584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.315950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cc0c0-5c3f-4712-911a-4c3f9883d2ab admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.321878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ad1fa-928c-4a30-9934-552503cca784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.323032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ad1fa-928c-4a30-9934-552503cca784 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.323453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ad1fa-928c-4a30-9934-552503cca784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.323995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ad1fa-928c-4a30-9934-552503cca784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.324809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ad1fa-928c-4a30-9934-552503cca784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.338128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ad1fa-928c-4a30-9934-552503cca784 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.347408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:40.359634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1133/2271] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:40.360485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.375873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:40.386854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac50a1b-1b11-46d8-8a64-870a49df410e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.393090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.394193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.394597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.395287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] PATH_INFO: `/v3/projects/e2f59ef5fc3a46a48b007659fd83722f/users/3220ab4f2d9c4a0a80c1b91397679b89/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.422397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.422732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.422963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.423340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f59ef5fc3a46a48b007659fd83722f, user_id=3220ab4f2d9c4a0a80c1b91397679b89, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.429806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.457783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1139/2272] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:40 2022] PUT /identity/v3/projects/e2f59ef5fc3a46a48b007659fd83722f/users/3220ab4f2d9c4a0a80c1b91397679b89/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:40.461441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.469296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] MFA Rules not processed for user `91bbba7d956d4390b66aaded767c1370`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:40.470534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:40.574704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fa094-61b1-4130-99c8-85db1a5ff9db admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.581099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.581486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.581766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.582083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.584668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.584668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.595767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1134/2273] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:40.604798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.607158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:40.620488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1140/2274] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:40.622793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.644845 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652386b8-1cd7-42f5-9a2b-498c227c4e5b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.678824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b8f4-763d-46d5-86ae-3ef8a2964236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.679842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b8f4-763d-46d5-86ae-3ef8a2964236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.680158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b8f4-763d-46d5-86ae-3ef8a2964236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.680437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b8f4-763d-46d5-86ae-3ef8a2964236 admin admin] PATH_INFO: `/v3/projects/e2f59ef5fc3a46a48b007659fd83722f/users/3220ab4f2d9c4a0a80c1b91397679b89/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.684539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529b8f4-763d-46d5-86ae-3ef8a2964236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f59ef5fc3a46a48b007659fd83722f, user_id=3220ab4f2d9c4a0a80c1b91397679b89, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.687377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529b8f4-763d-46d5-86ae-3ef8a2964236 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.708755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1141/2275] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:40 2022] PUT /identity/v3/projects/e2f59ef5fc3a46a48b007659fd83722f/users/3220ab4f2d9c4a0a80c1b91397679b89/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:40.722323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.722601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.722866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.755179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1afcf-ef6f-4707-8027-0e000f91c29a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.767615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] MFA Rules not processed for user `3220ab4f2d9c4a0a80c1b91397679b89`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:40.769377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:40.829470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.833100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.833375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.833657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.835701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:40.918004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1142/2276] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:40.932196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:40.936445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:40.945765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:40.948783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:40.950908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ecaf0-8361-445b-b37f-98c659e99387 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:40.951027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:40.955414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:40.956162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:40.956319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:40.956587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:40.963208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1135/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:46:41.017641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:41.041201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:41.042884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:41.053502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1143/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:42.553786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:42.565417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc76cb-c2cc-4a3f-bf51-72e8a1a0528b service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:42.573706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab884ef-3c09-4a13-8bdd-7b9d6da52c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:42.576732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab884ef-3c09-4a13-8bdd-7b9d6da52c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:42.577071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab884ef-3c09-4a13-8bdd-7b9d6da52c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:42.577352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab884ef-3c09-4a13-8bdd-7b9d6da52c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:42.712108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab884ef-3c09-4a13-8bdd-7b9d6da52c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:42.716952 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab884ef-3c09-4a13-8bdd-7b9d6da52c35 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:42.743731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1136/2279] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:43.724626 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:43.734188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aaf9b-6c36-44ab-a505-3cc4efa18423 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:43.737772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:43.738367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:43.738647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:43.738909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:43.801044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:43.802401 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:43.812998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1144/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:44.074284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:44.076636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:44.077148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:44.100373 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] MFA Rules not processed for user `5f294ab83780460f9679c7655c97a6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:44.102174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:44.183067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1137/2281] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:44.663906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:44.705718 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5a4f-338f-4a95-aa60-4721f79af6ee service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:44.715495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978beae-2732-4baa-be0d-2cddac17ae55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:44.716282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978beae-2732-4baa-be0d-2cddac17ae55 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:44.716696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978beae-2732-4baa-be0d-2cddac17ae55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:44.716980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978beae-2732-4baa-be0d-2cddac17ae55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:44.743284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978beae-2732-4baa-be0d-2cddac17ae55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:44.745185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978beae-2732-4baa-be0d-2cddac17ae55 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:44.767758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1145/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:49.081956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:49.186715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aab8ca-cdda-47a0-b889-9480dd155ce2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:49.206001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:49.211721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:49.212380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:49.213090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] PATH_INFO: `/v3/users/9d7db0c4b18c48b4966fa4cea76c06e0` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:49.214884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7db0c4b18c48b4966fa4cea76c06e0)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:49.221354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:49.322460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:49.341833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7db0c4b18c48b4966fa4cea76c06e0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:49.359725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7db0c4b18c48b4966fa4cea76c06e0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:49.361585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7db0c4b18c48b4966fa4cea76c06e0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:49.399391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7db0c4b18c48b4966fa4cea76c06e0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:49.403634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7db0c4b18c48b4966fa4cea76c06e0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:49.460387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1138/2283] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:49 2022] DELETE /identity/v3/users/9d7db0c4b18c48b4966fa4cea76c06e0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:49.466551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:49.467083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:49.467307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:49.519713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:49.523302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:49.678452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1146/2284] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:49.692340 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:49.694498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:49.779356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad7a20-543e-4bc8-aab8-1270fdd2ea16 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:49.783784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:49.784475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:49.784672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:49.784946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:49.794489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:49.795986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:49.806958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1139/2285] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:50.344179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:50.345741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:50.378115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46faa1-6085-430f-be86-9d7ca42614cc None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:50.392544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:50.393228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:50.393514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:50.393803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] PATH_INFO: `/v3/users/33070b58c99741eeb9f97a73507d012e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:50.408429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33070b58c99741eeb9f97a73507d012e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:50.410661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:50.494868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33070b58c99741eeb9f97a73507d012e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.495839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:50.497963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:50.525767 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33070b58c99741eeb9f97a73507d012e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.528191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33070b58c99741eeb9f97a73507d012e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.529677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33070b58c99741eeb9f97a73507d012e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.536807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c655b16-0bbb-45bf-b05f-2e8cd56caf38 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:50.552529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33070b58c99741eeb9f97a73507d012e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.558375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:50.572170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:50.573848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:50.575170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:50.576936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] PATH_INFO: `/v3/projects/22831dd6c00c490fb1afd86273c4b829` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:50.580042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22831dd6c00c490fb1afd86273c4b829)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:50.587259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:50.589316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1147/2286] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:50 2022] DELETE /identity/v3/users/33070b58c99741eeb9f97a73507d012e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:50.742946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22831dd6c00c490fb1afd86273c4b829', 'request_id': 'req-04a42843-cb1c-4960-99b7-d816318a3ef0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.755664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22831dd6c00c490fb1afd86273c4b829', 'request_id': 'req-04a42843-cb1c-4960-99b7-d816318a3ef0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.771871 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22831dd6c00c490fb1afd86273c4b829', 'request_id': 'req-04a42843-cb1c-4960-99b7-d816318a3ef0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.785484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] The token cache is being invalidate because project 22831dd6c00c490fb1afd86273c4b829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:50.786184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.787162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:50.789049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1140/2287] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:50 2022] DELETE /identity/v3/projects/22831dd6c00c490fb1afd86273c4b829 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:50.792896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:50.794339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:50.870060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f7fdb-1e93-4aa0-9c6c-6130e1cb0082 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:50.927184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:50.927924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:50.928117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:50.928383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] PATH_INFO: `/v3/users/481a4f2768934406b8ad1b710385357c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:50.929526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] RBAC: Authorizing `identity:delete_user(user_id=481a4f2768934406b8ad1b710385357c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:50.931547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:50.993790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481a4f2768934406b8ad1b710385357c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.003070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481a4f2768934406b8ad1b710385357c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.003794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481a4f2768934406b8ad1b710385357c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.004539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481a4f2768934406b8ad1b710385357c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.017473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.019382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:51.020585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d0263909-9028-4f51-af3f-c4e3532e565a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481a4f2768934406b8ad1b710385357c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.043882 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1148/2288] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:50 2022] DELETE /identity/v3/users/481a4f2768934406b8ad1b710385357c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:51.049919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a42843-cb1c-4960-99b7-d816318a3ef0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.139708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:51.169033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.169679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.169957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.170791 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] PATH_INFO: `/v3/projects/bc126ea1c77342d091a96baf33bedf6e` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.171379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc126ea1c77342d091a96baf33bedf6e)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.173392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.237352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc126ea1c77342d091a96baf33bedf6e', 'request_id': 'req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.261053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc126ea1c77342d091a96baf33bedf6e', 'request_id': 'req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.274223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc126ea1c77342d091a96baf33bedf6e', 'request_id': 'req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.296278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] The token cache is being invalidate because project bc126ea1c77342d091a96baf33bedf6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:51.297763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.299604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.303524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1141/2289] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:51 2022] DELETE /identity/v3/projects/bc126ea1c77342d091a96baf33bedf6e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:51.306554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d73a-57dc-41ac-b686-8e5d62017492 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.306843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d73a-57dc-41ac-b686-8e5d62017492 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.307135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d73a-57dc-41ac-b686-8e5d62017492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.337715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5b67d73a-57dc-41ac-b686-8e5d62017492 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:51.340333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b67d73a-57dc-41ac-b686-8e5d62017492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:51.350307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:51.421341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.422946 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:51.439421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1149/2290] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:51.447591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.447875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.448147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.482801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:51.485238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:51.497953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1150/2291] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:51.499434 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.507590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6eeaf-d71f-4bf1-bed6-fe8e3c6f8a61 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.509229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb9483-1d69-4657-9f9c-ba3908deca26 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.512684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.513387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.513944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.514039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] PATH_INFO: `/v3/projects/d0d4bed87a9545fb97166aee9b3b0489` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.514766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.515072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d4bed87a9545fb97166aee9b3b0489)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.515587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.516105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.516471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.517017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.517314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:51.517688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.519323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.539650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1151/2292] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:51.541684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.560900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719058e3-c680-4a2b-9846-f828954ace4b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.576739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.577363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.577643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.577915 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.578551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.581650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.607715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d4bed87a9545fb97166aee9b3b0489', 'request_id': 'req-cfa3b1c4-7340-4b13-94df-e3f36d182884'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.627824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d4bed87a9545fb97166aee9b3b0489', 'request_id': 'req-cfa3b1c4-7340-4b13-94df-e3f36d182884'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.640338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d4bed87a9545fb97166aee9b3b0489', 'request_id': 'req-cfa3b1c4-7340-4b13-94df-e3f36d182884'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.644471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1152/2293] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:51 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:51.648537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.667446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:51.679581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] The token cache is being invalidate because project d0d4bed87a9545fb97166aee9b3b0489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:51.680348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.681256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.687174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1142/2294] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:51 2022] DELETE /identity/v3/projects/d0d4bed87a9545fb97166aee9b3b0489 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:51.688301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.690259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:51.747261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e2867-5aea-477b-a7b9-6fa64a77951a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.761029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.761664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.763914 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.764304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.765080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.767255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.809036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1153/2295] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:51 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:51.813144 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.815136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:51.826580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3b1c4-7340-4b13-94df-e3f36d182884 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.829841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41748b6-c667-4c70-afcd-f47cd31b3aec admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.832861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.833987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.834515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.835157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.835567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] PATH_INFO: `/v3/users/3220ab4f2d9c4a0a80c1b91397679b89` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.835922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.836267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.836760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.837438 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.838279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3220ab4f2d9c4a0a80c1b91397679b89)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.841660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.841904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.859465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1154/2296] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:51.861490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.882151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1eb38-88fb-424a-8bdc-a2183b121e9a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.888052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.888853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.889040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.889386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.890039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.892049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:51.912208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1155/2297] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:51.916742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:51.918000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3220ab4f2d9c4a0a80c1b91397679b89'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.935431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3220ab4f2d9c4a0a80c1b91397679b89'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.936261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3220ab4f2d9c4a0a80c1b91397679b89'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.937446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce668ef-3f5b-4639-a050-6e56d148a4d0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:51.943345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6d391-e9c7-44a5-88d2-7a49a31d89ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:51.951600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6d391-e9c7-44a5-88d2-7a49a31d89ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:51.951600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6d391-e9c7-44a5-88d2-7a49a31d89ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:51.951600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6d391-e9c7-44a5-88d2-7a49a31d89ad admin admin] PATH_INFO: `/v3/projects/72d7dfe402c045edbca04c0e7eee05f9/users/36e8e45674624c219b59abb39d8dcf83/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:51.976990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3220ab4f2d9c4a0a80c1b91397679b89'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.979988 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6d391-e9c7-44a5-88d2-7a49a31d89ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d7dfe402c045edbca04c0e7eee05f9, user_id=36e8e45674624c219b59abb39d8dcf83, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:51.980294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3220ab4f2d9c4a0a80c1b91397679b89'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:51.984558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6d391-e9c7-44a5-88d2-7a49a31d89ad admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:52.009815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1156/2298] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:51 2022] PUT /identity/v3/projects/72d7dfe402c045edbca04c0e7eee05f9/users/36e8e45674624c219b59abb39d8dcf83/roles/87f1831257824560813f954348534540 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:46:52.012332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:52.013027 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:52.013995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:52.023864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1143/2299] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:51 2022] DELETE /identity/v3/users/3220ab4f2d9c4a0a80c1b91397679b89 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:52.078417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:52.079707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:52.087632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:52.255355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1157/2300] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:52.281050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:52.283445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:52.415085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f3c53-3bac-4f55-b906-9e75a1ceabb1 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:52.418966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3fb78-8712-4556-9dcf-c88e95f4c5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:52.419606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3fb78-8712-4556-9dcf-c88e95f4c5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:52.420349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3fb78-8712-4556-9dcf-c88e95f4c5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:52.420349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3fb78-8712-4556-9dcf-c88e95f4c5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:52.430975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3fb78-8712-4556-9dcf-c88e95f4c5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:52.432570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3fb78-8712-4556-9dcf-c88e95f4c5a5 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:52.443597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1144/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:52.806452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:52.807993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:52.821158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f574c-9093-445b-8773-8abb47570a72 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:52.829069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:52.829866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:52.830254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:52.830597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] PATH_INFO: `/v3/projects/e2f59ef5fc3a46a48b007659fd83722f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:52.831852 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f59ef5fc3a46a48b007659fd83722f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:52.833815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:52.885961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f59ef5fc3a46a48b007659fd83722f', 'request_id': 'req-d5761af0-77e2-4f46-9189-dd5443c3ac64'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:52.891986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f59ef5fc3a46a48b007659fd83722f', 'request_id': 'req-d5761af0-77e2-4f46-9189-dd5443c3ac64'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:52.904068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f59ef5fc3a46a48b007659fd83722f', 'request_id': 'req-d5761af0-77e2-4f46-9189-dd5443c3ac64'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:52.920745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] The token cache is being invalidate because project e2f59ef5fc3a46a48b007659fd83722f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:52.921032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:52.921679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d5761af0-77e2-4f46-9189-dd5443c3ac64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:52.923440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1158/2302] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:52 2022] DELETE /identity/v3/projects/e2f59ef5fc3a46a48b007659fd83722f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:52.928072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:52.928585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:52.928698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:52.948341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:52.949749 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:52.967236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:53.013326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1145/2303] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:53.022103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.022321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.022596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.043774 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:53.044976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:53.053054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1159/2304] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:53.057341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.070321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad0442-64fa-444a-8c6f-e5db78762dac None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.075774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.076220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.076621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.076993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.077839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:46:53.079083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.080041 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.104313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1146/2305] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:46:53.107571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.122638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a350-8c2e-44ff-92d6-3ca1d1dde423 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.128414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.129445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.130292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.130791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.131985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.135172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.181210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1160/2306] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:46:53 2022] POST /identity/v3/projects => generated 399 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:53.182166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.213439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333acec-0fc7-411d-a3d3-024ce0c22be6 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.314257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.315136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.315559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.315942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.316874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.318958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.352374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1147/2307] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:46:53 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:46:53.352572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.366381 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb987f5-f9a6-4a5a-9d02-2293f0f61538 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.373013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.373871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.374378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.375097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.376140 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.379278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.391512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1161/2308] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:46:53.395584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.408051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8dce7-73a4-4832-8571-b7aeef867f67 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.413052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6422d-2bd6-43f2-98f4-b2cd2ceb857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.413663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6422d-2bd6-43f2-98f4-b2cd2ceb857d admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.413950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6422d-2bd6-43f2-98f4-b2cd2ceb857d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.414221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6422d-2bd6-43f2-98f4-b2cd2ceb857d admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.414879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6422d-2bd6-43f2-98f4-b2cd2ceb857d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.416750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6422d-2bd6-43f2-98f4-b2cd2ceb857d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.429964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1148/2309] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:46:53.436725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.451601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97338cf0-23a4-4d2c-9dd3-38c803c3802e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.458727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.459616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.459894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.460707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] PATH_INFO: `/v3/projects/cea1579e13eb478b82855f8459fb78f4/users/ae396b1165914d0ab0d15872e7488bb6/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.485756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea1579e13eb478b82855f8459fb78f4, user_id=ae396b1165914d0ab0d15872e7488bb6, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.497302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.573861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1162/2310] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:46:53 2022] PUT /identity/v3/projects/cea1579e13eb478b82855f8459fb78f4/users/ae396b1165914d0ab0d15872e7488bb6/roles/87f1831257824560813f954348534540 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:53.583637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99464dbb-8572-4edd-8d77-33cde78ca8d9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.583978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99464dbb-8572-4edd-8d77-33cde78ca8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.584254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99464dbb-8572-4edd-8d77-33cde78ca8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.640818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-99464dbb-8572-4edd-8d77-33cde78ca8d9 None None] MFA Rules not processed for user `ae396b1165914d0ab0d15872e7488bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:53.642125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-99464dbb-8572-4edd-8d77-33cde78ca8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:53.736772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1149/2311] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:53.752994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:53.755550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:53.795028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:53.821707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ccf7f-13ac-4683-af1d-9962b5f5ffe2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:53.826879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:53.827555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:53.827872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:53.828191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:53.837118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:53.838547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:53.851084 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1163/2312] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:57.651930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:57.652521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:57.652820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:57.690576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] MFA Rules not processed for user `36e8e45674624c219b59abb39d8dcf83`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:46:57.692075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:57.697744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:57.790325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1150/2313] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:46:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:46:57.806532 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:57.808095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:57.921125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d30092-9524-46cf-a10a-b2293060134d service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:57.927530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:57.928143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:57.928984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:57.929261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:57.943020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:57.946134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:57.958505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1164/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:46:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:46:58.583774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:58.585237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:58.716912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c7ab1-c37a-40d8-a0db-28f9f5096759 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:58.725066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:58.725918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:58.726175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:58.726550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] PATH_INFO: `/v3/users/d30e9f0cee6a4fc2b5700ff5860098aa` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:58.728353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d30e9f0cee6a4fc2b5700ff5860098aa)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:58.731893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:58.857801 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd30e9f0cee6a4fc2b5700ff5860098aa'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:58.866328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd30e9f0cee6a4fc2b5700ff5860098aa'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:58.867453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd30e9f0cee6a4fc2b5700ff5860098aa'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:58.877725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd30e9f0cee6a4fc2b5700ff5860098aa'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:58.878353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd30e9f0cee6a4fc2b5700ff5860098aa'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:58.906975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1151/2315] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:58 2022] DELETE /identity/v3/users/d30e9f0cee6a4fc2b5700ff5860098aa => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:59.120035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:59.133014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926fc9c-c847-487c-b8aa-38423a22c92e service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:59.206137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:59.206770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:59.207221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:59.207511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] PATH_INFO: `/v3/projects/705ce7e25b8b4d3e9b60b20f9e6a9c06` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:59.210194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705ce7e25b8b4d3e9b60b20f9e6a9c06)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:59.212622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:59.262910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705ce7e25b8b4d3e9b60b20f9e6a9c06', 'request_id': 'req-31942515-f3fc-49f8-9cdc-883199eb1fa6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.269590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705ce7e25b8b4d3e9b60b20f9e6a9c06', 'request_id': 'req-31942515-f3fc-49f8-9cdc-883199eb1fa6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.272312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:59.278816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705ce7e25b8b4d3e9b60b20f9e6a9c06', 'request_id': 'req-31942515-f3fc-49f8-9cdc-883199eb1fa6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.290737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] The token cache is being invalidate because project 705ce7e25b8b4d3e9b60b20f9e6a9c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:46:59.291024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.291752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.293935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1165/2316] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:59 2022] DELETE /identity/v3/projects/705ce7e25b8b4d3e9b60b20f9e6a9c06 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:59.297831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:59.299282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:59.312033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a0bc-6deb-4faa-805f-d3bc58d4b75a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:59.342227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:46:59.420914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:59.421846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:59.422180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:59.422461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] PATH_INFO: `/v3/users/91bbba7d956d4390b66aaded767c1370` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:59.424404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91bbba7d956d4390b66aaded767c1370)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:59.427332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:59.504678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91bbba7d956d4390b66aaded767c1370'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.521650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91bbba7d956d4390b66aaded767c1370'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.523254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91bbba7d956d4390b66aaded767c1370'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.533978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91bbba7d956d4390b66aaded767c1370'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.534710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-80bf0e68-ede6-403f-baa1-43c144b4bb32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91bbba7d956d4390b66aaded767c1370'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:46:59.564111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1152/2317] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:46:59 2022] DELETE /identity/v3/users/91bbba7d956d4390b66aaded767c1370 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:46:59.768558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:46:59.770346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:46:59.815853 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31942515-f3fc-49f8-9cdc-883199eb1fa6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:46:59.905881 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:46:59.906801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:46:59.907282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:46:59.907877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] PATH_INFO: `/v3/projects/378daba287774c368fa20ce00f89e4df` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:46:59.909370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=378daba287774c368fa20ce00f89e4df)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:46:59.918291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:46:59.998822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '378daba287774c368fa20ce00f89e4df', 'request_id': 'req-a7f71542-2f03-4710-87e0-8b9f3c460ac2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:00.008526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '378daba287774c368fa20ce00f89e4df', 'request_id': 'req-a7f71542-2f03-4710-87e0-8b9f3c460ac2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:00.019660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921809-6567-43ee-a034-0492bafa48ef None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.020192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921809-6567-43ee-a034-0492bafa48ef None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.020192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921809-6567-43ee-a034-0492bafa48ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.024940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '378daba287774c368fa20ce00f89e4df', 'request_id': 'req-a7f71542-2f03-4710-87e0-8b9f3c460ac2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:00.038498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] The token cache is being invalidate because project 378daba287774c368fa20ce00f89e4df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:47:00.038793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:00.039449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7f71542-2f03-4710-87e0-8b9f3c460ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:00.041912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1166/2318] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:46:59 2022] DELETE /identity/v3/projects/378daba287774c368fa20ce00f89e4df => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:00.047906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.048189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.048465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.052192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-0d921809-6567-43ee-a034-0492bafa48ef None None] MFA Rules not processed for user `36e8e45674624c219b59abb39d8dcf83`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:00.054032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0d921809-6567-43ee-a034-0492bafa48ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:00.074023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:00.075894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:00.136109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1153/2319] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:47:00.152662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1167/2320] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:00.160848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.161135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.161406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.182659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:00.184559 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:00.193248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1154/2321] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:00.197709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.217129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1a7af-c602-4176-85a6-5d731885ab86 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.225565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.226855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.227299 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.227760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.229233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:47:00.229837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.232113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.248795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1168/2322] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:47:00.253286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.275723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a340-c875-4417-aed4-051b68e741b6 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.278787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.279397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.279708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.279983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.280627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.282534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.312162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1155/2323] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:00.315899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.326761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba7e2a-5093-4097-8957-d1e3bdfef406 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.330997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:00.391386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.392216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.392495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.392765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.393400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.395322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.423379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1169/2324] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:00.427362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.438589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad2ed-f10e-46fb-9537-a44592e51c73 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.443350 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.443984 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.444270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.444537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.445182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.446823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.460201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1156/2325] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:00.461536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.472131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7590a4f-32b4-4ef9-ae0b-02785fb9d46c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.476963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aeb0e-3038-4d35-b488-089cefad3d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.477669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aeb0e-3038-4d35-b488-089cefad3d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.478045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aeb0e-3038-4d35-b488-089cefad3d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.478411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aeb0e-3038-4d35-b488-089cefad3d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.479159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59aeb0e-3038-4d35-b488-089cefad3d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.481415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59aeb0e-3038-4d35-b488-089cefad3d39 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.493496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1170/2326] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:00.498007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.509008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2879e0-a34d-4352-be6a-56841c4cf91e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.514293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.515212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.515719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.516152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] PATH_INFO: `/v3/projects/68f9b3e544914f63bcf5e6f072dd2576/users/94ddfd7e42664ac0bb1af63f782e1d62/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.527102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:00.553534 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f9b3e544914f63bcf5e6f072dd2576, user_id=94ddfd7e42664ac0bb1af63f782e1d62, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.556094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.583605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1157/2327] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:47:00 2022] PUT /identity/v3/projects/68f9b3e544914f63bcf5e6f072dd2576/users/94ddfd7e42664ac0bb1af63f782e1d62/roles/87f1831257824560813f954348534540 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:00.585360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.585674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.585923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.626684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] MFA Rules not processed for user `94ddfd7e42664ac0bb1af63f782e1d62`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:00.627987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:00.719749 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1171/2328] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:00.720605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.795628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31307b-85c5-4916-a99b-972cbeb47672 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.800993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.801658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.801992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.802266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.802909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.805243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.839960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1158/2329] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:00.845161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:00.877966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f114-f326-40c9-9451-c144f13211cf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:00.955863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:00.956798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:00.957149 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:00.957488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:00.958267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:00.960623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:00.992759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1172/2330] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:47:00 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:00.994019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.012388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10097c61-3965-41f9-88ef-e957ab5a6257 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.032323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.033008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.033354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.033692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.034944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.037058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.050320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1159/2331] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:01.054102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.062700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:01.074610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08d3de-0b6f-4161-a367-65bd8c956fa2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.081400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625d226-5623-4e65-a5b4-30ea978a6e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.082367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625d226-5623-4e65-a5b4-30ea978a6e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.083193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625d226-5623-4e65-a5b4-30ea978a6e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.083959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625d226-5623-4e65-a5b4-30ea978a6e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.084541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625d226-5623-4e65-a5b4-30ea978a6e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.086293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625d226-5623-4e65-a5b4-30ea978a6e6d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.098581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1173/2332] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:01.102908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.113739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f23a-c959-4ca8-93de-5fa064ca89d4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.118860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.119587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.119869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.120309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] PATH_INFO: `/v3/projects/8d959089456c44d19ab8014f4a7fa757/users/c7ac0182c292464e8cec2f48fc3983fb/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.136341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d959089456c44d19ab8014f4a7fa757, user_id=c7ac0182c292464e8cec2f48fc3983fb, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.139218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.175539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1160/2333] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:47:01 2022] PUT /identity/v3/projects/8d959089456c44d19ab8014f4a7fa757/users/c7ac0182c292464e8cec2f48fc3983fb/roles/87f1831257824560813f954348534540 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:01.178886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.178886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.179292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.216078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] MFA Rules not processed for user `c7ac0182c292464e8cec2f48fc3983fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:01.217684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:01.316703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1174/2334] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:01.321075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.421999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f41e-1459-4bab-9b10-d3436558a919 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.428449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.429538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.429538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.429983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.430738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.432948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.487655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1161/2335] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:47:01 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:01.487895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.515873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b4d5a-363d-473a-989d-02d3f9140227 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.588925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.590071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.590344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.590614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.591239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.593207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.622292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1175/2336] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:47:01 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:47:01.623580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.634537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e27f37-c49f-4cac-8dbe-8424746deb80 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.639560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.640261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.640640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.641007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.641772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.643471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.662834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1162/2337] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:01.667066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.678797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca397-a711-4d34-821f-5d13a6465e36 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.684164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.684895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.685180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.685754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] PATH_INFO: `/v3/projects/89b3fc52eada4068946b505d08f3039c/users/600936499c2f4a68811755a79a63aa5e/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.702849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b3fc52eada4068946b505d08f3039c, user_id=600936499c2f4a68811755a79a63aa5e, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.705582 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.722466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1176/2338] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:47:01 2022] PUT /identity/v3/projects/89b3fc52eada4068946b505d08f3039c/users/600936499c2f4a68811755a79a63aa5e/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:47:01.726443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.738074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bee1e-3243-4947-9471-8dff19350a90 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.746999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:01.810540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907e739-b41c-4c64-99d6-f9eb05976e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.811313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907e739-b41c-4c64-99d6-f9eb05976e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.811762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907e739-b41c-4c64-99d6-f9eb05976e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.812130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907e739-b41c-4c64-99d6-f9eb05976e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.812883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907e739-b41c-4c64-99d6-f9eb05976e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.814938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907e739-b41c-4c64-99d6-f9eb05976e6e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.833435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1163/2339] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:01.837271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:01.851263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311da7e-45a2-4996-8606-d19640f10f63 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:01.857153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:01.857925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.858276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.858622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] PATH_INFO: `/v3/projects/89b3fc52eada4068946b505d08f3039c/users/600936499c2f4a68811755a79a63aa5e/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.861463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b3fc52eada4068946b505d08f3039c, user_id=600936499c2f4a68811755a79a63aa5e, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:01.864494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:01.894458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1177/2340] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:47:01 2022] PUT /identity/v3/projects/89b3fc52eada4068946b505d08f3039c/users/600936499c2f4a68811755a79a63aa5e/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:01.896355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:01.896796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:01.897147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:01.930985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] MFA Rules not processed for user `600936499c2f4a68811755a79a63aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:01.932166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:02.026831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1164/2341] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:02.058239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:02.060395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:02.112638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:02.132099 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e1b91-1afe-47f4-839a-322d3230f09e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:02.136742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:02.137372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:02.137650 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:02.137919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:02.197897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:02.200006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:02.209161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1178/2342] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:04.852168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:04.854034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:04.876052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed81e5-90c2-4958-9c44-3be5389cf05a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:04.880096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:04.881578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:04.882038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:04.882378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:04.962938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:04.964421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:04.975617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1165/2343] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:05.453556 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:05.454993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:05.540753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481dff73-07f8-4c78-ae6e-b73f383aa740 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:05.545863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:05.546506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:05.546896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:05.547378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] PATH_INFO: `/v3/users/ae396b1165914d0ab0d15872e7488bb6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:05.548527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae396b1165914d0ab0d15872e7488bb6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:05.550896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:05.605120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae396b1165914d0ab0d15872e7488bb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:05.622635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae396b1165914d0ab0d15872e7488bb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:05.623294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae396b1165914d0ab0d15872e7488bb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:05.624230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae396b1165914d0ab0d15872e7488bb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:05.629118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:05.640559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ba09e798-a377-4b45-a3d8-404f7c5bf411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae396b1165914d0ab0d15872e7488bb6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:05.643647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db781d-5bae-4751-83c7-61de8fa40589 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:05.652413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:05.653073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:05.653348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:05.653612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:05.655375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:05.661346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1179/2344] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:47:05 2022] DELETE /identity/v3/users/ae396b1165914d0ab0d15872e7488bb6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:05.667599 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:05.667973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:05.668242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:05.671740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:05.673172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:05.682987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1166/2345] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:47:05.697041 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:05.698256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:05.703019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:05.770336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1180/2346] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:47:05.780873 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:05.782388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:05.810330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:05.847828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d54267-284a-4c11-82ab-fe724c97047e service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:05.897174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:05.897877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:05.898057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:05.898361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:05.907863 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:05.909296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:05.918524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1167/2347] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:06.272693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:06.274504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:06.289399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fd8fc-e99d-4249-b606-263745f48f64 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:06.294927 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:06.295859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:06.296142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:06.296414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] PATH_INFO: `/v3/projects/cea1579e13eb478b82855f8459fb78f4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:06.297571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea1579e13eb478b82855f8459fb78f4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:06.300860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:06.355330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea1579e13eb478b82855f8459fb78f4', 'request_id': 'req-f311b4ed-0a32-4433-b17d-6fa0f44361ee'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:06.361581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea1579e13eb478b82855f8459fb78f4', 'request_id': 'req-f311b4ed-0a32-4433-b17d-6fa0f44361ee'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:06.370331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea1579e13eb478b82855f8459fb78f4', 'request_id': 'req-f311b4ed-0a32-4433-b17d-6fa0f44361ee'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:06.381615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] The token cache is being invalidate because project cea1579e13eb478b82855f8459fb78f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:47:06.381903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:06.382520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f311b4ed-0a32-4433-b17d-6fa0f44361ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:06.384037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1181/2348] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:47:06 2022] DELETE /identity/v3/projects/cea1579e13eb478b82855f8459fb78f4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:15.914466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=38 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:16.378289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:20.287461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:20.288992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:20.302423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e293658-fbf0-41c6-ba96-c1f0a6042565 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:20.358051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:20.358670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:20.358945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:20.359213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] PATH_INFO: `/v3/users/94ddfd7e42664ac0bb1af63f782e1d62` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:20.360347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ddfd7e42664ac0bb1af63f782e1d62)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:20.362247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:20.409349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ddfd7e42664ac0bb1af63f782e1d62'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:20.418821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ddfd7e42664ac0bb1af63f782e1d62'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:20.419442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ddfd7e42664ac0bb1af63f782e1d62'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:20.427162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ddfd7e42664ac0bb1af63f782e1d62'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:20.427817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ddfd7e42664ac0bb1af63f782e1d62'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:20.445389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1168/2349] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:47:20 2022] DELETE /identity/v3/users/94ddfd7e42664ac0bb1af63f782e1d62 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:20.450181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:20.450648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:20.450899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:20.474324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:20.475537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:20.539414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1182/2350] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:20.551631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:20.553095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:20.611235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c2283-03bf-4855-bb8a-f8195036c154 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:20.615049 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:20.615698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:20.615970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:20.616237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:20.626107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:20.627551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:20.637317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1169/2351] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:21.001240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:21.002839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:21.017310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8f1b3-87a7-427c-a7fc-e6528ce4d3c2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:21.022085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:21.023312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:21.023641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:21.024105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] PATH_INFO: `/v3/projects/68f9b3e544914f63bcf5e6f072dd2576` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:21.025415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f9b3e544914f63bcf5e6f072dd2576)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:21.027244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:21.077040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f9b3e544914f63bcf5e6f072dd2576', 'request_id': 'req-86a14956-78ac-430b-bf49-6042891eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.082720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f9b3e544914f63bcf5e6f072dd2576', 'request_id': 'req-86a14956-78ac-430b-bf49-6042891eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.090179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f9b3e544914f63bcf5e6f072dd2576', 'request_id': 'req-86a14956-78ac-430b-bf49-6042891eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.101828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] The token cache is being invalidate because project 68f9b3e544914f63bcf5e6f072dd2576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:47:21.102113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.102722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.104205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1183/2352] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:47:21 2022] DELETE /identity/v3/projects/68f9b3e544914f63bcf5e6f072dd2576 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:21.108280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:21.117847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf2896-237f-4c8f-a9c3-3b3386930a09 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:21.136121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:21.180084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:21.180722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:21.181006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:21.181275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] PATH_INFO: `/v3/users/c7ac0182c292464e8cec2f48fc3983fb` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:21.182419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ac0182c292464e8cec2f48fc3983fb)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:21.184393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:21.231940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ac0182c292464e8cec2f48fc3983fb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.239269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ac0182c292464e8cec2f48fc3983fb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.239823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ac0182c292464e8cec2f48fc3983fb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.246716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ac0182c292464e8cec2f48fc3983fb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.247334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ac0182c292464e8cec2f48fc3983fb'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.265310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1170/2353] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:47:21 2022] DELETE /identity/v3/users/c7ac0182c292464e8cec2f48fc3983fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:21.561889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:21.571497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:21.577581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14956-78ac-430b-bf49-6042891eac58 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:21.637812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:21.638462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:21.638801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:21.639612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] PATH_INFO: `/v3/projects/8d959089456c44d19ab8014f4a7fa757` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:21.640730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d959089456c44d19ab8014f4a7fa757)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:21.642669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:21.677998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:21.696335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d959089456c44d19ab8014f4a7fa757', 'request_id': 'req-fd418c80-afd9-40fc-b24a-4e196fbfa31a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.702092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d959089456c44d19ab8014f4a7fa757', 'request_id': 'req-fd418c80-afd9-40fc-b24a-4e196fbfa31a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.709452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d959089456c44d19ab8014f4a7fa757', 'request_id': 'req-fd418c80-afd9-40fc-b24a-4e196fbfa31a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.720773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] The token cache is being invalidate because project 8d959089456c44d19ab8014f4a7fa757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:47:21.721062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.721701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.724517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1184/2354] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:47:21 2022] DELETE /identity/v3/projects/8d959089456c44d19ab8014f4a7fa757 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:21.728488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:21.729911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:21.743388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e76f7-c5e1-4828-93df-000adca7e40d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:21.801528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:21.802149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:21.802430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:21.802707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] PATH_INFO: `/v3/users/600936499c2f4a68811755a79a63aa5e` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:21.803851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600936499c2f4a68811755a79a63aa5e)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:21.805766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:21.850001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:21.853616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600936499c2f4a68811755a79a63aa5e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.861784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600936499c2f4a68811755a79a63aa5e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.862409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600936499c2f4a68811755a79a63aa5e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.869230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600936499c2f4a68811755a79a63aa5e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.869858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-307b924d-e518-4c92-9136-d0af9d09abb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600936499c2f4a68811755a79a63aa5e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:21.889031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1171/2355] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:47:21 2022] DELETE /identity/v3/users/600936499c2f4a68811755a79a63aa5e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:22.271215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.272815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:22.285911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd418c80-afd9-40fc-b24a-4e196fbfa31a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.342583 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.343209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.343501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.343781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] PATH_INFO: `/v3/projects/89b3fc52eada4068946b505d08f3039c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.344886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b3fc52eada4068946b505d08f3039c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.346791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.394981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b3fc52eada4068946b505d08f3039c', 'request_id': 'req-e99429d5-bb13-426c-a2d9-01eed159ebe8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:22.400990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b3fc52eada4068946b505d08f3039c', 'request_id': 'req-e99429d5-bb13-426c-a2d9-01eed159ebe8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:22.408653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b3fc52eada4068946b505d08f3039c', 'request_id': 'req-e99429d5-bb13-426c-a2d9-01eed159ebe8'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:22.419505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] The token cache is being invalidate because project 89b3fc52eada4068946b505d08f3039c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:47:22.419787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:22.421026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e99429d5-bb13-426c-a2d9-01eed159ebe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:22.422469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1185/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:47:22 2022] DELETE /identity/v3/projects/89b3fc52eada4068946b505d08f3039c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:22.427171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.427452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.427759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.446745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:22.447973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:22.507650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1172/2357] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:22.515965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.516251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.516529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.544566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:22.546112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:22.554871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1186/2358] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:22.559363 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.571346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0d99d-b95d-49f4-b9a7-4606ac30a7db None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.581514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.582126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.582402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.582789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.583308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:47:22.583662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.585770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.601331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1173/2359] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:47:22.605310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.615930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c37d3-a54b-4f09-a53a-2ac6349ffc25 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.621627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.624394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.624673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.624941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.625584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.627586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.658917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1187/2360] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:22.664058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.678794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498a01a-fdaf-4cf6-be66-159d8b9b5d71 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.734204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.734890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.735071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.735337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.736044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.737959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.758211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1174/2361] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:47:22 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:47:22.762483 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.773464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ec7a-b724-4150-8cb2-f1136e7a3f39 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.778961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.780137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.780416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.780683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.781322 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.783168 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.794090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1188/2362] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:22.798052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.808654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac76c8-28f6-4d3f-b230-033d539d02a3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.813857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b2ce8-1061-4e88-b913-06a8a9d41454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.814451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b2ce8-1061-4e88-b913-06a8a9d41454 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.814726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b2ce8-1061-4e88-b913-06a8a9d41454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.814993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b2ce8-1061-4e88-b913-06a8a9d41454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.815659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b2ce8-1061-4e88-b913-06a8a9d41454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.817241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b2ce8-1061-4e88-b913-06a8a9d41454 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.828084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1175/2363] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:22.832163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:22.838368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:22.842177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84580b5-4345-44d8-873f-f05bc2c40a23 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:22.850338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:22.850999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.851283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.852028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] PATH_INFO: `/v3/projects/14678b3a1af541249eb221af8d5bfd95/users/03d10bae2f474e31aea189e51430d128/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.868375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14678b3a1af541249eb221af8d5bfd95, user_id=03d10bae2f474e31aea189e51430d128, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:22.871021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:22.883954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1189/2364] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:47:22 2022] PUT /identity/v3/projects/14678b3a1af541249eb221af8d5bfd95/users/03d10bae2f474e31aea189e51430d128/roles/87f1831257824560813f954348534540 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:47:22.953841 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53535d1b-db7f-4cd8-a5f7-a777a7fb1e48 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:22.954116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53535d1b-db7f-4cd8-a5f7-a777a7fb1e48 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:22.954385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53535d1b-db7f-4cd8-a5f7-a777a7fb1e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:22.981692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-53535d1b-db7f-4cd8-a5f7-a777a7fb1e48 None None] MFA Rules not processed for user `03d10bae2f474e31aea189e51430d128`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:22.982914 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-53535d1b-db7f-4cd8-a5f7-a777a7fb1e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:22.995104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:23.049695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1176/2365] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:47:23.062254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:23.063955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:23.119083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cfbd3-c886-4a1f-afe2-707173e80f17 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:23.122704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:23.123842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:23.124118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:23.124386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:23.132784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:23.134197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:23.143376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1190/2366] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:33.046618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=20 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:33.138833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:36.294153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd43c5-6af4-4885-b23f-59533388fd7c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:36.294783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd43c5-6af4-4885-b23f-59533388fd7c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:36.294783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd43c5-6af4-4885-b23f-59533388fd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:36.314597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-17cd43c5-6af4-4885-b23f-59533388fd7c None None] MFA Rules not processed for user `4e0e6a33679f4cac96704f71dcfa5c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:36.315849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-17cd43c5-6af4-4885-b23f-59533388fd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:36.392769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1177/2367] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:43.425085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:43.426919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:43.514808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffce148-ac60-4241-817d-297df5beab42 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:43.521164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:43.521947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:43.522997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:43.523323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:43.525221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:43.538716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:43.540620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:43.558870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1191/2368] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:46.377637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=25 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:50.650364 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:50.651009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:50.651009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:50.670137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:50.671356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:50.725880 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1178/2369] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:47:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:50.767629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:50.769518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:50.835576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a520e1d-300b-4840-baf0-591d0a50e824 service placement] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:50.840520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf954c-6bb6-4931-9a8f-a0962baff3ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:50.841113 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf954c-6bb6-4931-9a8f-a0962baff3ba service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:50.841391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf954c-6bb6-4931-9a8f-a0962baff3ba service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:50.841662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf954c-6bb6-4931-9a8f-a0962baff3ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:50.849459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf954c-6bb6-4931-9a8f-a0962baff3ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:50.851016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf954c-6bb6-4931-9a8f-a0962baff3ba service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:50.858776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1192/2370] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:47:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:47:52.445377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:52.446789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:52.516420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a0eed-1db5-4ea8-9dc3-f783b399d700 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:52.520886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:52.521503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:52.521778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:52.522043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] PATH_INFO: `/v3/users/36bfc3d59b9a4d68b9fb8a71b521ec02` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:52.523164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bfc3d59b9a4d68b9fb8a71b521ec02)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:52.525109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:52.588670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bfc3d59b9a4d68b9fb8a71b521ec02'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:52.598698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bfc3d59b9a4d68b9fb8a71b521ec02'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:52.599461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bfc3d59b9a4d68b9fb8a71b521ec02'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:52.608223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bfc3d59b9a4d68b9fb8a71b521ec02'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:52.609448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bfc3d59b9a4d68b9fb8a71b521ec02'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:52.637929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1179/2371] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:47:52 2022] DELETE /identity/v3/users/36bfc3d59b9a4d68b9fb8a71b521ec02 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:52.642466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:52.652337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:52.653794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:52.682405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:52.683863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:52.766244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1193/2372] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:47:52.777322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:52.781135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:52.833097 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:52.932633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152fa76-8f36-43fa-9d98-92b3b5615f16 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:52.938519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c5075-aadf-423c-8379-cbd608804d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:52.939285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c5075-aadf-423c-8379-cbd608804d12 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:52.939823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c5075-aadf-423c-8379-cbd608804d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:52.940511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c5075-aadf-423c-8379-cbd608804d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:52.953088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c5075-aadf-423c-8379-cbd608804d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:52.954511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c5075-aadf-423c-8379-cbd608804d12 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:52.966384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1180/2373] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:47:53.287475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.289275 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:53.335359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e2ffd-171f-48a5-a7c6-6e83042f81a4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.342153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.342780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.343051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.343316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] PATH_INFO: `/v3/projects/43d65129e3bf46a68d154912738cf99f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.344498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d65129e3bf46a68d154912738cf99f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.347708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.366905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:53.414156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d65129e3bf46a68d154912738cf99f', 'request_id': 'req-61692070-8c02-4906-82be-0ced95e904f9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:53.422690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d65129e3bf46a68d154912738cf99f', 'request_id': 'req-61692070-8c02-4906-82be-0ced95e904f9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:53.432548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d65129e3bf46a68d154912738cf99f', 'request_id': 'req-61692070-8c02-4906-82be-0ced95e904f9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:53.447308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] The token cache is being invalidate because project 43d65129e3bf46a68d154912738cf99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:47:53.447568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:53.448593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-61692070-8c02-4906-82be-0ced95e904f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:47:53.455227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1194/2374] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:47:53 2022] DELETE /identity/v3/projects/43d65129e3bf46a68d154912738cf99f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:53.457069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.457795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.457795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.482169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:53.483495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:53.553025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1181/2375] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:47:53.561955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.561955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.562291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.585048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:53.585506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:53.595531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1195/2376] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:53.599243 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.609303 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd608-34d3-4ae1-b1eb-5de1c9093f7c None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.614448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.615081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.615359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.615667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.616324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:47:53.616622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.618264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.633333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1182/2377] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:47:53.638250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.653777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9425d46-e9c4-41d1-963c-67548f1717f9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.659990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.661022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.661324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.661625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.662348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.664480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.700808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1196/2378] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:47:53 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:47:53.704657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.716042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c4e8-676c-482f-a4f2-f80ac3354f2c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.786929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.788120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.788554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.788999 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.789757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.791843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.817739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1183/2379] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:47:53 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:47:53.821827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.834456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4d28c-6179-44c6-81d4-f36af570d2d5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.840966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.841586 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.841866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.842394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.843336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.845205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.860731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1197/2380] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:47:53.862778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.880998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9505d-dbab-436b-b5e0-c7b3f356faac admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.901615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4da3b-344f-49b8-a08d-293ce5beb7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.902450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4da3b-344f-49b8-a08d-293ce5beb7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.903549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4da3b-344f-49b8-a08d-293ce5beb7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.904001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4da3b-344f-49b8-a08d-293ce5beb7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.904658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4da3b-344f-49b8-a08d-293ce5beb7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.906285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4da3b-344f-49b8-a08d-293ce5beb7b4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.919213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1184/2381] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:47:53.924458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:53.942849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af3bf1-925b-4116-a900-f2bce6bd1db4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:53.947963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:53.948731 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.949013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.949286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] PATH_INFO: `/v3/projects/743501da05424df197d7af03d32eabb3/users/65f9c7fc74c448829608dab23d7023ce/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:53.966695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743501da05424df197d7af03d32eabb3, user_id=65f9c7fc74c448829608dab23d7023ce, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:53.969351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:53.984047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1198/2382] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:47:53 2022] PUT /identity/v3/projects/743501da05424df197d7af03d32eabb3/users/65f9c7fc74c448829608dab23d7023ce/roles/87f1831257824560813f954348534540 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:47:53.990421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9ed01-add6-474c-b332-ca67dce8f30d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:53.990892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9ed01-add6-474c-b332-ca67dce8f30d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:53.991166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9ed01-add6-474c-b332-ca67dce8f30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:54.011934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-3dc9ed01-add6-474c-b332-ca67dce8f30d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:47:54.013696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3dc9ed01-add6-474c-b332-ca67dce8f30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:54.046787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:47:54.113606 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1185/2383] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:47:54.119993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:47:54.122153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:47:54.207292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9618d7-0f10-425b-b06c-8eb4836ca175 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:47:54.212029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:47:54.212648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:47:54.212920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:47:54.213187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:47:54.225235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:47:54.226603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:47:54.237702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1199/2384] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:48:00.516650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:00.518070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:00.518352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:00.589546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] MFA Rules not processed for user `65f9c7fc74c448829608dab23d7023ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:00.593247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:00.725335 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1186/2385] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:00.726127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:00.739243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb53ad-6013-474f-8f7b-cdbd84bd780b service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:00.745780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:00.746437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:00.746711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:00.746998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:00.748710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:00.750598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:00.808543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1200/2386] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:48:00 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:48:00.812497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:00.832830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7d79c-9ace-45d5-88b3-fb73423e2788 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:00.942682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:00.944819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:00.945489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:00.946069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:00.947318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:00.952163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:00.978459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1187/2387] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:48:00 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:48:00.983186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:00.998823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96a7b-29c1-44c1-83f3-6c52d5e29695 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:01.006753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:01.007383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:01.007993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:01.008392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:01.009201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:01.010930 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:01.033092 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:01.044154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1201/2388] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:48:01.052291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bd30b-666b-4d91-9218-bab258d3ba80 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:01.069823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:01.083189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:01.083508 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:01.083779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] PATH_INFO: `/v3/projects/2fdc30ed86ab4859ad1987f99cf009f0/users/705d7b5148ac4a81a5ad8a6c071f38ad/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:01.149205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdc30ed86ab4859ad1987f99cf009f0, user_id=705d7b5148ac4a81a5ad8a6c071f38ad, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:01.152021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:01.219368 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:01.238754 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1188/2389] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:48:01 2022] PUT /identity/v3/projects/2fdc30ed86ab4859ad1987f99cf009f0/users/705d7b5148ac4a81a5ad8a6c071f38ad/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:48:01.245771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ec28c-008f-4678-998e-caf6209f13f7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:01.266504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:01.469933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c200a95-d4ee-41df-8e4c-97d9ddbc4378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:01.472710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200a95-d4ee-41df-8e4c-97d9ddbc4378 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:01.473005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200a95-d4ee-41df-8e4c-97d9ddbc4378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:01.473646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200a95-d4ee-41df-8e4c-97d9ddbc4378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:01.474441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c200a95-d4ee-41df-8e4c-97d9ddbc4378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:01.478727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c200a95-d4ee-41df-8e4c-97d9ddbc4378 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:01.505391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1202/2390] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:48:01.509800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:01.523567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdf53-62b6-4881-b6a3-e33dbbde7403 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:01.530684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:01.531906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:01.532342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:01.532819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] PATH_INFO: `/v3/projects/2fdc30ed86ab4859ad1987f99cf009f0/users/705d7b5148ac4a81a5ad8a6c071f38ad/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:01.535078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdc30ed86ab4859ad1987f99cf009f0, user_id=705d7b5148ac4a81a5ad8a6c071f38ad, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:01.538266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:01.557358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1189/2391] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:48:01 2022] PUT /identity/v3/projects/2fdc30ed86ab4859ad1987f99cf009f0/users/705d7b5148ac4a81a5ad8a6c071f38ad/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:48:08.157611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51b03-f095-4a7e-a266-3cbb9df42907 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:08.158257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51b03-f095-4a7e-a266-3cbb9df42907 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:08.158257 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51b03-f095-4a7e-a266-3cbb9df42907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:08.198088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0ba51b03-f095-4a7e-a266-3cbb9df42907 None None] MFA Rules not processed for user `705d7b5148ac4a81a5ad8a6c071f38ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:08.200063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0ba51b03-f095-4a7e-a266-3cbb9df42907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:08.321403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1203/2392] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:08.338450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:08.341098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:08.373787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:08.483991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765a32-cf92-4164-a445-7db5ad5c6209 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:08.492582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76707be-0ebf-4ad8-bac4-d104ec2a243f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:08.494044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76707be-0ebf-4ad8-bac4-d104ec2a243f service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:08.494702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76707be-0ebf-4ad8-bac4-d104ec2a243f service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:08.495369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76707be-0ebf-4ad8-bac4-d104ec2a243f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:08.609760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76707be-0ebf-4ad8-bac4-d104ec2a243f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:08.611921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76707be-0ebf-4ad8-bac4-d104ec2a243f service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:08.627081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1190/2393] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:48:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:48:10.282612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:10.283440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:10.283977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:10.307076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] MFA Rules not processed for user `65f9c7fc74c448829608dab23d7023ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:10.308673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:10.326073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1204/2394] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:18.618664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=49 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:20.299780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=62 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:25.139059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:25.139545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:25.139643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:25.159453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:25.160690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:25.215773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1191/2395] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 00:48:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:25.228212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:25.229620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:25.299409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ac277-fc0a-4fc0-b29d-0adcf08bd066 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:25.304233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:25.304822 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:25.305095 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:25.305360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:25.313689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:25.315248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:25.322941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1205/2396] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:48:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:48:31.506291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:31.507808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:31.573377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb70cad-c7d3-4578-bff4-2ba0526277b1 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:31.579231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:31.579888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:31.580162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:31.580430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] PATH_INFO: `/v3/users/5f294ab83780460f9679c7655c97a6e4` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:31.581583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f294ab83780460f9679c7655c97a6e4)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:31.583535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:31.629667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f294ab83780460f9679c7655c97a6e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.636406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f294ab83780460f9679c7655c97a6e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.637524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f294ab83780460f9679c7655c97a6e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.644879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f294ab83780460f9679c7655c97a6e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.645510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-14fb4a29-89ed-445a-89f6-2f90d4928681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f294ab83780460f9679c7655c97a6e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.662166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1192/2397] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:48:31 2022] DELETE /identity/v3/users/5f294ab83780460f9679c7655c97a6e4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:48:31.844647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:31.856720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410cb69-fc79-4bc1-9d51-edc54bab5191 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:31.916112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:31.917000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:31.917283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:31.918106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] PATH_INFO: `/v3/projects/06a0252a4fba4f1ebfa9f25a2c6e39a4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:31.919261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a0252a4fba4f1ebfa9f25a2c6e39a4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:31.921225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:31.977046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a0252a4fba4f1ebfa9f25a2c6e39a4', 'request_id': 'req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.982609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a0252a4fba4f1ebfa9f25a2c6e39a4', 'request_id': 'req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:31.989550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a0252a4fba4f1ebfa9f25a2c6e39a4', 'request_id': 'req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:32.001277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] The token cache is being invalidate because project 06a0252a4fba4f1ebfa9f25a2c6e39a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:48:32.001564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:32.002218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a1f1c407-acc0-46bf-971a-884bdaf5c7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:48:32.004010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1206/2398] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:48:31 2022] DELETE /identity/v3/projects/06a0252a4fba4f1ebfa9f25a2c6e39a4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:48:32.008918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.009209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.009487 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.029498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:32.030750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:32.049166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:32.092006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1193/2399] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:32.099736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.100020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.100289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.120121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:32.121931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:32.129971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1207/2400] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:32.134052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.143793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8ca4-aed8-48cb-bc98-83e4af3113ab None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.148540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.149131 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.149409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.149679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.150331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:48:32.150633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.152246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.163799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1194/2401] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:48:32.168959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.180030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11423949-d8dc-4796-ac20-8884cf08fd8d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.185585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.186197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.186475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.186755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.187377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.189319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.200924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:32.220662 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1208/2402] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:48:32 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:48:32.225225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.239146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9ef07-a145-4408-bc8f-98618b3dc245 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.296862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.297460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.297735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.298005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.298668 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.300664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.320709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1195/2403] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:48:32 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:48:32.324947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.335857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c93d8-b6cb-4dff-a9fa-b2402ffd7f25 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.342090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.342709 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.342990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.343264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.343961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.345614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.356903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1209/2404] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:48:32.361201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.371066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dbd83-a5af-44ff-935e-04510402da73 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.375788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe77cd5-7ca5-49d7-86a5-c8e7f0d9c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.376374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe77cd5-7ca5-49d7-86a5-c8e7f0d9c213 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.376652 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe77cd5-7ca5-49d7-86a5-c8e7f0d9c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.377012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe77cd5-7ca5-49d7-86a5-c8e7f0d9c213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.377543 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe77cd5-7ca5-49d7-86a5-c8e7f0d9c213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.379388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe77cd5-7ca5-49d7-86a5-c8e7f0d9c213 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.391959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1196/2405] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:48:32.396895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.409223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436b373-3d59-4896-a0cb-c113b85870b5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.414192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.415087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.415454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.415865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] PATH_INFO: `/v3/projects/2a313f2139a841baa6f299c931a49cab/users/be5d59ad4e6940048c3bb0fd0a415d51/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.432158 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a313f2139a841baa6f299c931a49cab, user_id=be5d59ad4e6940048c3bb0fd0a415d51, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.435000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.448403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1210/2406] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:48:32 2022] PUT /identity/v3/projects/2a313f2139a841baa6f299c931a49cab/users/be5d59ad4e6940048c3bb0fd0a415d51/roles/87f1831257824560813f954348534540 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:48:32.459382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277414f-da37-4ec2-8eea-e4333f894ba6 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.459698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277414f-da37-4ec2-8eea-e4333f894ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.459969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277414f-da37-4ec2-8eea-e4333f894ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.490475 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5277414f-da37-4ec2-8eea-e4333f894ba6 None None] MFA Rules not processed for user `be5d59ad4e6940048c3bb0fd0a415d51`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:32.491721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5277414f-da37-4ec2-8eea-e4333f894ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:32.559322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1197/2407] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:48:32.571839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:48:32.573539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:32.631354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439305e-3fac-4534-9b66-97af1c48cf62 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:48:32.634979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:48:32.635591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:32.635865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:32.637070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:32.645591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:48:32.647047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:48:32.656776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1211/2408] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:48:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:48:42.555587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:42.652398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:48:56.848592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f764f1-cd72-4042-88b8-c62c06466d99 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:48:56.849068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f764f1-cd72-4042-88b8-c62c06466d99 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:48:56.849181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f764f1-cd72-4042-88b8-c62c06466d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:48:56.867670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-17f764f1-cd72-4042-88b8-c62c06466d99 None None] MFA Rules not processed for user `b8e4bda3331540d1a0b869270db9c25d`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:48:56.868849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-17f764f1-cd72-4042-88b8-c62c06466d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:48:56.921068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1198/2409] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:48:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:01.557342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:01.559240 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:01.624608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842895-1faf-4ad7-8d93-29299ddd1947 service glance] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:01.629409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:01.630019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:01.630292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:01.630559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] PATH_INFO: `/v3/users/03d10bae2f474e31aea189e51430d128` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:01.631690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d10bae2f474e31aea189e51430d128)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:01.633613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:01.680648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d10bae2f474e31aea189e51430d128'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:01.688963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d10bae2f474e31aea189e51430d128'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:01.689810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d10bae2f474e31aea189e51430d128'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:01.690423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d10bae2f474e31aea189e51430d128'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:01.697981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d10bae2f474e31aea189e51430d128'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:01.715891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1212/2410] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:49:01 2022] DELETE /identity/v3/users/03d10bae2f474e31aea189e51430d128 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:01.720591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:01.721081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:01.721212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:01.745645 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:01.746865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:01.810115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1199/2411] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:01.820032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:01.821404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:01.880178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9162e-c2ca-43e6-8066-00b45e852cdf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:01.884062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8597-250c-44d0-a9ce-91fbc3b1b9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:01.884688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8597-250c-44d0-a9ce-91fbc3b1b9ed service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:01.884994 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8597-250c-44d0-a9ce-91fbc3b1b9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:01.885267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8597-250c-44d0-a9ce-91fbc3b1b9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:01.894989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f8597-250c-44d0-a9ce-91fbc3b1b9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:01.896505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f8597-250c-44d0-a9ce-91fbc3b1b9ed service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:01.906348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1213/2412] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:02.248068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.249471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:02.263628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e79e7-d00d-45a6-9ac9-34410b1b455e None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.268493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.269158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.269436 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.269734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] PATH_INFO: `/v3/projects/14678b3a1af541249eb221af8d5bfd95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.270890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=14678b3a1af541249eb221af8d5bfd95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.273046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.320555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14678b3a1af541249eb221af8d5bfd95', 'request_id': 'req-85ccad56-262d-41b2-a77a-47ff37a8f4dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:02.326199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14678b3a1af541249eb221af8d5bfd95', 'request_id': 'req-85ccad56-262d-41b2-a77a-47ff37a8f4dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:02.333828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14678b3a1af541249eb221af8d5bfd95', 'request_id': 'req-85ccad56-262d-41b2-a77a-47ff37a8f4dd'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:02.345145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] The token cache is being invalidate because project 14678b3a1af541249eb221af8d5bfd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:49:02.345432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:02.346050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-85ccad56-262d-41b2-a77a-47ff37a8f4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:02.347540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1200/2413] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:49:02 2022] DELETE /identity/v3/projects/14678b3a1af541249eb221af8d5bfd95 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:02.352417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.352705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.352981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.372417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:02.373609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:02.390371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:02.433909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1214/2414] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:02.442794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.443078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.443352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.462410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:02.464143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:02.471898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1201/2415] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:02.475843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.486076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b6de-af93-4d77-a696-85fa5562aaf8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.490789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.491379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.491684 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.491953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.492596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:49:02.492896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.494477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.506169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1215/2416] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:49:02.510200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.519404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6171688-bb14-4f51-bbce-902ba9963c6d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.524123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.524721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.524997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.525263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.526387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.528301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.552605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:02.572073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1202/2417] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:49:02 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:02.576729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.586919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691b94a-0bb3-43e8-8049-408474d7e114 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.644630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.645276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.645563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.645835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.646453 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.648414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.669751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1216/2418] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:49:02 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:02.673935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.684509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18e22-56f6-42c4-bc2f-91acd86d5d4c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.689070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.689656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.689931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.690198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.690831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.692440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.702550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1203/2419] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:02.706358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.716732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da17d27-94f6-4a96-b20b-46c74bc4e2bc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.721306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b71f6-d3d4-4aca-9b6b-80c65fa8cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.721884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b71f6-d3d4-4aca-9b6b-80c65fa8cb76 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.722157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b71f6-d3d4-4aca-9b6b-80c65fa8cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.722423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b71f6-d3d4-4aca-9b6b-80c65fa8cb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.723057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b71f6-d3d4-4aca-9b6b-80c65fa8cb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.724732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b71f6-d3d4-4aca-9b6b-80c65fa8cb76 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.735332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1217/2420] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:02.744311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.754773 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7fde5-00b6-4780-9d95-e2077cea982c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.759324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.760494 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.760770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.761040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] PATH_INFO: `/v3/projects/53d8d76185c44a159439c0f7a0e88b31/users/d308262224af475091e6aa494ea839b1/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.774884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d8d76185c44a159439c0f7a0e88b31, user_id=d308262224af475091e6aa494ea839b1, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.778200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.790788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1204/2421] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:49:02 2022] PUT /identity/v3/projects/53d8d76185c44a159439c0f7a0e88b31/users/d308262224af475091e6aa494ea839b1/roles/87f1831257824560813f954348534540 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:02.796049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c2b28-8f86-4915-b2fa-f700d57e8e52 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.796335 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c2b28-8f86-4915-b2fa-f700d57e8e52 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.796611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c2b28-8f86-4915-b2fa-f700d57e8e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.814961 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ab0c2b28-8f86-4915-b2fa-f700d57e8e52 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:02.816192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab0c2b28-8f86-4915-b2fa-f700d57e8e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:02.873601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1218/2422] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:02.883602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:02.885789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:02.941066 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f618ea-d1b3-43ca-9141-6b3c489bea1d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:02.944654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:02.945274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:02.945544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:02.945809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:02.954998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:02.956434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:02.966737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1205/2423] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:07.216835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.217365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.217502 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.245187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] MFA Rules not processed for user `d308262224af475091e6aa494ea839b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:07.246400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:07.264218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:07.314250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1219/2424] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:07.318739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:07.328295 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f15cb3-dcc4-4c7c-b3dd-41348e843bc2 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:07.332913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:07.333502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.333779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.334050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.334690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:07.336622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:07.369979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1206/2425] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:49:07 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:07.374078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:07.384983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49548cdc-cf7b-4670-9b04-ff339cdb3ba4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:07.445057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:07.445673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.445955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.446228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.446869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:07.448885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:07.470867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1220/2426] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:49:07 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:07.474886 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:07.485840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d11a7-a2d7-4b08-af5f-f859e01373d5 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:07.490431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:07.491050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.491326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.491658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.492300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:07.494017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:07.505045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1207/2427] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:07.509251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:07.520198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafacd4-c9fc-4a7c-8823-de213ffbfe5c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:07.524981 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:07.526265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.526545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.526817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] PATH_INFO: `/v3/projects/eb9e29ab32b9438ca3c1680a93d1ac17/users/2822b030129549c09a33b160f5aa6965/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.542219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9e29ab32b9438ca3c1680a93d1ac17, user_id=2822b030129549c09a33b160f5aa6965, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:07.545083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:07.564567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1221/2428] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:49:07 2022] PUT /identity/v3/projects/eb9e29ab32b9438ca3c1680a93d1ac17/users/2822b030129549c09a33b160f5aa6965/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:07.568724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:07.579382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d139b53-b1ff-4397-b83e-6f8782194e7f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:07.686860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92dbd1-f442-4061-b262-194d3be345e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:07.687775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92dbd1-f442-4061-b262-194d3be345e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.688075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92dbd1-f442-4061-b262-194d3be345e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.688353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92dbd1-f442-4061-b262-194d3be345e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.689146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92dbd1-f442-4061-b262-194d3be345e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:07.690957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92dbd1-f442-4061-b262-194d3be345e3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:07.701944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1208/2429] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:07.706059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:07.718902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671404-41b2-40e8-beec-1ef465795ef5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:07.723681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:07.724342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:07.724621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:07.724895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] PATH_INFO: `/v3/projects/eb9e29ab32b9438ca3c1680a93d1ac17/users/2822b030129549c09a33b160f5aa6965/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:07.729191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9e29ab32b9438ca3c1680a93d1ac17, user_id=2822b030129549c09a33b160f5aa6965, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:07.732508 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:07.743488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1222/2430] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:49:07 2022] PUT /identity/v3/projects/eb9e29ab32b9438ca3c1680a93d1ac17/users/2822b030129549c09a33b160f5aa6965/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:13.841953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:13.842450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:13.842704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:13.884925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] MFA Rules not processed for user `2822b030129549c09a33b160f5aa6965`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:13.886211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:13.973900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:14.169497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1209/2431] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:14.176490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:14.179052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:14.301698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4495f-6c0d-4d38-a111-2c4dfa02b26d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:14.317869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:14.320294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:14.320811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:14.321093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:14.329002 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:14.416234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:14.418212 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:14.432107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1223/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:24.150155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=20 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:24.426573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=24 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:29.543696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:29.596575 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56213a0-99d8-4206-837b-f10f9fbd32ab None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:29.600139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:29.600729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:29.601004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:29.601283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:29.603074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:29.617154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:29.618560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:29.630046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1210/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:30.220068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:30.221493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:30.238377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb35e8-3ac5-4074-bf7f-b57926ef1324 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:30.242051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:30.242648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:30.242925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:30.243192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:30.244948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:30.258455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:30.259919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:30.270557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1224/2434] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:38.453645 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:38.464797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f89d-3833-464b-b2ad-23b9f41c343c service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:38.471035 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:38.471655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:38.471929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:38.472195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:38.473955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:38.530186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:38.531788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f58ffb-fa1f-4ff4-b3af-37a61c93d833 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:38.541119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1211/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:38.795836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:38.797262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:38.862062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1d00-fa9a-47eb-bda7-6ac6f5f973f4 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:38.866601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:38.867216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:38.867522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:38.867796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] PATH_INFO: `/v3/users/be5d59ad4e6940048c3bb0fd0a415d51` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:38.869300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5d59ad4e6940048c3bb0fd0a415d51)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:38.871209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:38.917073 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5d59ad4e6940048c3bb0fd0a415d51'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:38.925545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5d59ad4e6940048c3bb0fd0a415d51'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:38.926161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5d59ad4e6940048c3bb0fd0a415d51'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:38.926808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5d59ad4e6940048c3bb0fd0a415d51'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:38.933958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5d59ad4e6940048c3bb0fd0a415d51'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:38.951392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1225/2436] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:49:38 2022] DELETE /identity/v3/users/be5d59ad4e6940048c3bb0fd0a415d51 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:38.955822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:38.956134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:38.956408 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:38.975028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:38.976251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:39.034162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1212/2437] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:39.043924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.045636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:39.106162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201de1-e524-40f2-aa73-a69be04b434d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.109768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07298e-c022-4aa1-a550-747d5dfd3020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.110370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07298e-c022-4aa1-a550-747d5dfd3020 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.110647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07298e-c022-4aa1-a550-747d5dfd3020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.111451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07298e-c022-4aa1-a550-747d5dfd3020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.120476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07298e-c022-4aa1-a550-747d5dfd3020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.121843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07298e-c022-4aa1-a550-747d5dfd3020 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.132498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1226/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:39.398521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.400028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:39.416747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42c639-74bb-4332-8bfa-5ffc275ec7a3 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.421379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.422095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.422273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.422542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] PATH_INFO: `/v3/projects/2a313f2139a841baa6f299c931a49cab` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.424265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a313f2139a841baa6f299c931a49cab)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.426130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.462375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:39.472536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a313f2139a841baa6f299c931a49cab', 'request_id': 'req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:39.478229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a313f2139a841baa6f299c931a49cab', 'request_id': 'req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:39.484818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a313f2139a841baa6f299c931a49cab', 'request_id': 'req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:39.496440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] The token cache is being invalidate because project 2a313f2139a841baa6f299c931a49cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:49:39.496726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:39.497356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c0da7a8b-9413-4fe2-9e5e-e3652cd7e5b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:49:39.498830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1213/2439] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:49:39 2022] DELETE /identity/v3/projects/2a313f2139a841baa6f299c931a49cab => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:39.504000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.504300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.504571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.523909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:39.525081 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:39.529157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:39.586004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1227/2440] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:39.593889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.594181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.594460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.614661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:39.615920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:39.623825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1214/2441] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:39.628250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.637870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b168d1-ec07-4c75-8720-cf6ee3890152 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.642998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.643574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.643856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.644128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.644776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:49:39.645080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.646705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.658911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1228/2442] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:49:39.663084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.673741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee04c1-3b40-43a7-a20b-3e145a1fddc3 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.679790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.680393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.680667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.680934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.681570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.683471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.712313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1215/2443] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:49:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:39.716488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.732124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a784c-40c3-4d47-8116-362c626fe6e3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.797197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.797883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.798121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.798410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.799044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.801038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.822864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1229/2444] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:49:39 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:39.827051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.837797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980887b1-b1d5-4d58-873c-79153cf7eac7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.842441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.843029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.843304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.843595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.844234 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.846474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.859145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1216/2445] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:49:39.860390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.870816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3005dbc-f0c9-49bf-b29e-70bd21804281 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.875821 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319104-e6c9-4d0d-b291-44de628ea34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.876947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319104-e6c9-4d0d-b291-44de628ea34f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.877232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319104-e6c9-4d0d-b291-44de628ea34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.877505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319104-e6c9-4d0d-b291-44de628ea34f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.878420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319104-e6c9-4d0d-b291-44de628ea34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.880117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319104-e6c9-4d0d-b291-44de628ea34f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.890867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1230/2446] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:39.895795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:39.905476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc06224-9da1-4d94-aae5-d9a0dce2bbdf admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:39.910064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:39.910729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.911013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.911284 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] PATH_INFO: `/v3/projects/762beb87caad4aa691528a7d9de01086/users/5d47be4ab52f4034ac09fbf043990709/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.927213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762beb87caad4aa691528a7d9de01086, user_id=5d47be4ab52f4034ac09fbf043990709, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:39.930161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:39.943461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1217/2447] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:49:39 2022] PUT /identity/v3/projects/762beb87caad4aa691528a7d9de01086/users/5d47be4ab52f4034ac09fbf043990709/roles/87f1831257824560813f954348534540 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:39.948891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31718f-cf5e-4448-8379-a3de37b660e9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:39.949172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31718f-cf5e-4448-8379-a3de37b660e9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:39.949442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31718f-cf5e-4448-8379-a3de37b660e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:39.968872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-db31718f-cf5e-4448-8379-a3de37b660e9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:39.970132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-db31718f-cf5e-4448-8379-a3de37b660e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:40.027876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1231/2448] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:40.037506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:40.039179 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:40.093857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7e0b-32aa-42f4-bcf9-0242061b8862 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:40.097477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:40.098060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:40.098334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:40.098600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:40.107496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:40.108926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:40.118518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1218/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:44.676485 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617bb19-696a-45c9-b693-759607836435 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:44.676955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617bb19-696a-45c9-b693-759607836435 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:44.677055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617bb19-696a-45c9-b693-759607836435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:44.708358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c617bb19-696a-45c9-b693-759607836435 None None] MFA Rules not processed for user `5d47be4ab52f4034ac09fbf043990709`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:44.709560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c617bb19-696a-45c9-b693-759607836435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:44.713738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:44.778271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1232/2450] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:44.782894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:44.793136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91e35f-1891-4b21-85a6-a18ac3d2bd32 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:44.798004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:44.799125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:44.799405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:44.799712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:44.800344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:44.802232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:44.831392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1219/2451] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:49:44 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:44.835272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617bb19-696a-45c9-b693-759607836435 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:44.847558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617bb19-696a-45c9-b693-759607836435 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:44.914993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:44.915663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:44.916078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:44.916355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:44.917001 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:44.919120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:44.947131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1233/2452] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:49:44 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:49:44.951227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:44.962391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f6558-aa17-4616-a768-7536b07d30df admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:44.967354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:44.968088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:44.968568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:44.968909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:44.969645 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:44.971334 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:44.990149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1220/2453] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:44.991455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:45.005056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2cb6-c26c-4445-bffd-8fe8f80656e9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:45.011860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:45.012939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:45.013221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:45.013518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] PATH_INFO: `/v3/projects/0293f640bfc245fa9f271bfcd15a9036/users/b2a3d16e2c01431e857eaead6bd70b07/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:45.031607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0293f640bfc245fa9f271bfcd15a9036, user_id=b2a3d16e2c01431e857eaead6bd70b07, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:45.034330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:45.052539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1234/2454] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:49:44 2022] PUT /identity/v3/projects/0293f640bfc245fa9f271bfcd15a9036/users/b2a3d16e2c01431e857eaead6bd70b07/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:49:45.056485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:45.085057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e909ad-c9f5-4544-89db-e1aa224dd3c9 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:45.120845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:45.152866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3144e-b1e3-4c16-8055-a15311d1ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:45.153476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3144e-b1e3-4c16-8055-a15311d1ec77 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:45.153753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3144e-b1e3-4c16-8055-a15311d1ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:45.154025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3144e-b1e3-4c16-8055-a15311d1ec77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:45.154711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3144e-b1e3-4c16-8055-a15311d1ec77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:45.156338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3144e-b1e3-4c16-8055-a15311d1ec77 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:45.167742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1221/2455] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:49:45.170873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:45.182364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a13a1-edb9-4714-a442-920c16630e6d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:45.187202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:45.187895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:45.188177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:45.188451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] PATH_INFO: `/v3/projects/0293f640bfc245fa9f271bfcd15a9036/users/b2a3d16e2c01431e857eaead6bd70b07/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:45.191191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0293f640bfc245fa9f271bfcd15a9036, user_id=b2a3d16e2c01431e857eaead6bd70b07, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:45.193895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:45.207205 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1235/2456] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:49:45 2022] PUT /identity/v3/projects/0293f640bfc245fa9f271bfcd15a9036/users/b2a3d16e2c01431e857eaead6bd70b07/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:49:50.038807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad9ce0-7eba-411a-a962-2c8cdb56e17d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:50.039327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad9ce0-7eba-411a-a962-2c8cdb56e17d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:50.039327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad9ce0-7eba-411a-a962-2c8cdb56e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:50.067108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-23ad9ce0-7eba-411a-a962-2c8cdb56e17d None None] MFA Rules not processed for user `b2a3d16e2c01431e857eaead6bd70b07`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:50.068432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-23ad9ce0-7eba-411a-a962-2c8cdb56e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:50.154663 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1222/2457] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:50.165432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:50.167215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:50.241477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcd02-5f25-4970-9ed3-20c696fc646d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:50.245736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:50.246356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:50.246633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:50.247421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:50.260187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:50.328252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:50.329801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:50.340837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1236/2458] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:50.604723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:50.605226 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:50.605683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:50.630637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:50.630637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:50.702093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1223/2459] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:49:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:50.759447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:50.761058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:50.838462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51f9da-8eb7-4c51-9b7a-3556ec53edb1 service glance] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:50.844717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba2331-8ae4-480b-9769-a943e8ba8188 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:50.845645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba2331-8ae4-480b-9769-a943e8ba8188 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:50.845985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba2331-8ae4-480b-9769-a943e8ba8188 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:50.846323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba2331-8ae4-480b-9769-a943e8ba8188 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:50.859595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba2331-8ae4-480b-9769-a943e8ba8188 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:50.859595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba2331-8ae4-480b-9769-a943e8ba8188 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:50.868866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1237/2460] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:49:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:49:53.354403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:53.356804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:53.460844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff0d00-207b-41e7-9adf-8c70174def89 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:53.465069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:53.466092 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:53.466533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:53.466962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:53.468953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:53.482441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:53.484246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:53.490108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:53.522848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1224/2461] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:54.924949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:54.925546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:54.925546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:54.954062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] MFA Rules not processed for user `d308262224af475091e6aa494ea839b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:49:54.965340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:55.091526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1238/2462] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:49:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:49:55.508442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:55.519714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fd228-5b2a-471e-bd12-7e12df2c0baa service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:55.525502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:55.527139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:55.527750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:55.528676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:55.531312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:55.543750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:55.545469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590176a-bcb0-449e-afcc-9141d8df98d2 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:55.554518 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1225/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:49:56.343148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:49:56.344964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:56.436030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:49:56.471380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0e204-a47c-407f-afe9-7eb08bc5c657 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:49:56.478307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:49:56.478950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:49:56.479224 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:49:56.479504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:49:56.481378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:49:56.494713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:49:56.496472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:49:56.507653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1239/2464] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:49:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:01.507774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:01.508245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:01.508339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:01.527881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] MFA Rules not processed for user `5d47be4ab52f4034ac09fbf043990709`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:50:01.529069 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:01.536506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1226/2465] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:50:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:50:06.502781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=11 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:50:09.748312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:09.750608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:09.866115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:09.876781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee601ac-3d04-4662-afb3-256015b12647 service placement] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:09.880325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a4508-46b7-437a-9551-ef0cf640419d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:09.888086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:09.888602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:09.889187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:09.889619 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:09.889899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:09.890445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:09.890985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:09.891500 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:09.892621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:09.894681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:09.910015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:09.912924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73312bd-911c-4ef0-8443-051fd6dbcf21 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:09.972279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1240/2466] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:09.974312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:09.992198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:10.014134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1227/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:10.275732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:10.276024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:10.276305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:10.313352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:50:10.316245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:10.326445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1241/2468] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:50:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:50:10.341552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:10.355969 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac28b23-dfd3-41c9-abe5-1046f41f7eef service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:10.362028 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:10.362765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:10.363109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:10.363512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:10.373057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:10.374951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:10.385542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1228/2469] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:50:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:50:20.300543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=37 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:50:20.381122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=45 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:50:37.956635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:37.958252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:37.975197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcb7a0-a71e-49e5-959d-3c5131abf328 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:37.980698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:37.981294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:37.981571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:37.981847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:37.984198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:38.008328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:38.009989 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:38.021992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1242/2470] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:48.017725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:50:50.953795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:50.963005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955dce1-3628-400a-a89a-cb036f29abda service swift] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:50.966625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:50.967207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:50.967507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:50.967795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:50.969511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:51.031274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:51.032660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:51.042017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1229/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:52.625170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:52.638044 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292c187-461b-4aa6-aec2-abb6ef2e0f92 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:52.648461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:52.649086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:52.649366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:52.649643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:52.651722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:52.664752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:52.666331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:52.680121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1243/2472] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:53.396704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:53.409963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3185fe-e88d-43c9-8c8f-aaf53ba90f47 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:53.415844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e558-df31-4655-984a-540d85866273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:53.416397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e558-df31-4655-984a-540d85866273 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:53.416682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e558-df31-4655-984a-540d85866273 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:53.416921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e558-df31-4655-984a-540d85866273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:53.418692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e223e558-df31-4655-984a-540d85866273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:50:53.480101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223e558-df31-4655-984a-540d85866273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:53.481747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223e558-df31-4655-984a-540d85866273 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:53.492281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1230/2473] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:50:56.769707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:50:56.784848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d7f6e-31fc-4099-ad65-ea49d5ac512f service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:50:56.788819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:50:56.789455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:50:56.789735 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:50:56.790004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:50:56.800172 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:50:56.801591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:50:56.811834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1244/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:50:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:51:03.487298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=58 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:05.223080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e558-df31-4655-984a-540d85866273 service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:05.234900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e558-df31-4655-984a-540d85866273 service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:05.240859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:05.241536 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:05.241884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:05.242227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:05.244599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:05.257505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:05.259205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:05.272658 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1231/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:51:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:51:05.327441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:05.328871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:05.401638 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571fa70-32b4-471a-b26c-329b93978ac7 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:05.406856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:05.408064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:05.408342 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:05.408613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] PATH_INFO: `/v3/users/b8e4bda3331540d1a0b869270db9c25d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:05.409760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e4bda3331540d1a0b869270db9c25d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:05.411760 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:05.457219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e4bda3331540d1a0b869270db9c25d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.464163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e4bda3331540d1a0b869270db9c25d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.464777 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e4bda3331540d1a0b869270db9c25d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.465374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e4bda3331540d1a0b869270db9c25d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.473146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-59f5be18-7e7d-4889-8a31-5a2c13d48563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e4bda3331540d1a0b869270db9c25d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.490793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1245/2476] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:51:05 2022] DELETE /identity/v3/users/b8e4bda3331540d1a0b869270db9c25d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:51:05.676391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:05.692211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012f232-1fb4-4419-ba6a-798fe50b3a20 service placement] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:05.748704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:05.749348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:05.749619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:05.749888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] PATH_INFO: `/v3/projects/39a872cd1b1e451ba5786fe246950f73` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:05.750987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a872cd1b1e451ba5786fe246950f73)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:05.753273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:05.799040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a872cd1b1e451ba5786fe246950f73', 'request_id': 'req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.804394 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a872cd1b1e451ba5786fe246950f73', 'request_id': 'req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.811601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a872cd1b1e451ba5786fe246950f73', 'request_id': 'req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.822220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] The token cache is being invalidate because project 39a872cd1b1e451ba5786fe246950f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:51:05.822505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.823142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:51:05.824628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1232/2477] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:51:05 2022] DELETE /identity/v3/projects/39a872cd1b1e451ba5786fe246950f73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:51:15.487215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:15.819326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:23.199824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189f84f-7bd4-425a-b741-3e1bd38909c2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:23.200359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189f84f-7bd4-425a-b741-3e1bd38909c2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:23.200472 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189f84f-7bd4-425a-b741-3e1bd38909c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:23.219751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4189f84f-7bd4-425a-b741-3e1bd38909c2 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:51:23.220953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4189f84f-7bd4-425a-b741-3e1bd38909c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:23.275249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1246/2478] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:51:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:51:23.315774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:23.317229 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:23.388222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2a3d5-a5ab-43a1-926c-1667fe1495bf admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:23.393208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:23.393805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:23.394082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:23.394352 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:23.402360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:23.403985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:23.413240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1233/2479] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:51:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:51:25.367366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a18d3c-4ad2-44ca-a91e-588b1b24d9a8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:25.368048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a18d3c-4ad2-44ca-a91e-588b1b24d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:25.368598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a18d3c-4ad2-44ca-a91e-588b1b24d9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:25.395803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b0a18d3c-4ad2-44ca-a91e-588b1b24d9a8 None None] MFA Rules not processed for user `4e0e6a33679f4cac96704f71dcfa5c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:51:25.397018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b0a18d3c-4ad2-44ca-a91e-588b1b24d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:25.459702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1247/2480] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:51:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:51:33.409560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=32 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:35.451350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=50 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:38.647034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:38.648537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:38.703735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef39dd-f6d4-400a-8b15-121ca0a63036 service swift] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:38.707898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:38.708488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:38.708761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:38.709026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:38.710718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:38.719926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:38.721367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d51bd-5fb6-4d5f-b94c-6a411e40367c service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:38.729071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1234/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:51:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:51:39.864382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206b1e-0b6b-44f2-b2e2-f4608775802d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:39.864911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206b1e-0b6b-44f2-b2e2-f4608775802d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:39.864911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206b1e-0b6b-44f2-b2e2-f4608775802d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:39.866535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1248/2482] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:51:39.872740 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc965-afab-45ae-9cb3-7bd629171bcd None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:39.873178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc965-afab-45ae-9cb3-7bd629171bcd None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:39.873470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc965-afab-45ae-9cb3-7bd629171bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:39.894247 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-688dc965-afab-45ae-9cb3-7bd629171bcd None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:51:39.895447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-688dc965-afab-45ae-9cb3-7bd629171bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:39.968152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1235/2483] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:51:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:51:39.981375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206b1e-0b6b-44f2-b2e2-f4608775802d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:39.983107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1c206b1e-0b6b-44f2-b2e2-f4608775802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:40.040694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206b1e-0b6b-44f2-b2e2-f4608775802d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:40.044249 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66944-d800-4729-a753-a015b6705b2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:40.044840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66944-d800-4729-a753-a015b6705b2c service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.045118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66944-d800-4729-a753-a015b6705b2c service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.045388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66944-d800-4729-a753-a015b6705b2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.055210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66944-d800-4729-a753-a015b6705b2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:40.056674 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66944-d800-4729-a753-a015b6705b2c service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:40.067126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1249/2484] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:51:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:51:40.140153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f1513-a801-4322-bb83-98cd52145253 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.140435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f1513-a801-4322-bb83-98cd52145253 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.140712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f1513-a801-4322-bb83-98cd52145253 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.142065 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1236/2485] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:51:40.147325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b554cd-1ad0-4406-9edf-1f034113e76e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.147648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b554cd-1ad0-4406-9edf-1f034113e76e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.147943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b554cd-1ad0-4406-9edf-1f034113e76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.167410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-87b554cd-1ad0-4406-9edf-1f034113e76e None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:51:40.168621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-87b554cd-1ad0-4406-9edf-1f034113e76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:40.174948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1250/2486] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:51:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:51:40.186321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f1513-a801-4322-bb83-98cd52145253 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:40.197556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f1513-a801-4322-bb83-98cd52145253 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:40.201044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c45713-87da-447c-8f09-def566397802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:40.201632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c45713-87da-447c-8f09-def566397802 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.201906 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c45713-87da-447c-8f09-def566397802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.202175 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c45713-87da-447c-8f09-def566397802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.209713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c45713-87da-447c-8f09-def566397802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:40.211127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c45713-87da-447c-8f09-def566397802 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:40.218735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1237/2487] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:51:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:51:40.299388 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cb07a-8db2-4466-ac95-b97d80008456 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.299703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cb07a-8db2-4466-ac95-b97d80008456 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.299976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cb07a-8db2-4466-ac95-b97d80008456 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.301319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1251/2488] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:51:40.306610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f38a58-65fb-454e-a92a-9565426cfd2a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.306896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f38a58-65fb-454e-a92a-9565426cfd2a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.307169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f38a58-65fb-454e-a92a-9565426cfd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.326827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-42f38a58-65fb-454e-a92a-9565426cfd2a None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:51:40.328057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-42f38a58-65fb-454e-a92a-9565426cfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:40.337847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1238/2489] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:51:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:51:40.347899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cb07a-8db2-4466-ac95-b97d80008456 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:40.358595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cb07a-8db2-4466-ac95-b97d80008456 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:40.362316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:40.362903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:40.363179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:40.363449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:40.373496 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:40.374920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:40.386175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1252/2490] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:51:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:51:43.253470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:43.253993 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:43.254090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:43.272959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:51:43.274169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:43.281840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1239/2491] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:51:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:51:43.294866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:43.306598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0eb5b-bbad-4f29-82d6-9f0fe5ab37b3 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:43.311603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c42d54-22d1-45a9-9f15-5106622aeea8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:43.312192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c42d54-22d1-45a9-9f15-5106622aeea8 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:43.312598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c42d54-22d1-45a9-9f15-5106622aeea8 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:43.312752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c42d54-22d1-45a9-9f15-5106622aeea8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:43.321017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c42d54-22d1-45a9-9f15-5106622aeea8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:43.322790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c42d54-22d1-45a9-9f15-5106622aeea8 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:43.330802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1253/2492] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:51:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:51:53.267562 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=69 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:53.327673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=41 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:51:58.835214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:51:58.836765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:58.853012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25188135-d1ed-4340-80f0-d7f32ca6e795 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:51:58.858598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:51:58.859201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:51:58.859506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:51:58.859774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:51:58.862143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:51:58.920359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:51:58.921934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:51:58.931843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1240/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:51:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:52:08.927405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=28 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:52:11.623966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:11.624431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:11.624531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:11.643266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:52:11.644489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:11.651605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1254/2494] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 00:52:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:52:11.664385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:52:11.676390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9a97-cf3f-4172-9f55-5ba1a255038e service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:52:11.681258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:52:11.681848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:11.682127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:11.682398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:11.690420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:52:11.692017 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:52:11.699850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1241/2495] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:52:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:52:21.638433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:52:21.696664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=6 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:52:44.936900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:52:44.938379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:45.091847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd296-bb9a-4dba-9764-1605459fa9e6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:52:45.098023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:52:45.098023 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:45.098258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:45.098495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:45.100289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:45.115103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:52:45.117141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e86ab8-689b-4daa-ad3b-c46695ab0ecf service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:52:45.148303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1255/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:52:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:52:55.124566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=63 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:52:56.285882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:52:56.314826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd6a6-d053-4546-a613-e80582b19b28 service swift] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:52:56.318459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:52:56.319050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:56.319326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:56.319628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:56.321355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:56.404506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:52:56.405922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7e96-153c-4ba0-b4a6-bb1ac9ac115e service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:52:56.424681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1242/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:52:59.581531 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840bba1-84f7-4943-a547-c9b3623fe385 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:59.582011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840bba1-84f7-4943-a547-c9b3623fe385 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:59.582266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840bba1-84f7-4943-a547-c9b3623fe385 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:59.583813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1256/2498] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:52:59.595316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:59.595867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:59.595867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:59.656811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] MFA Rules not processed for user `ce70c358cf0f46139f419cf894a97fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:52:59.658430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:59.663268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840bba1-84f7-4943-a547-c9b3623fe385 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:52:59.664903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e840bba1-84f7-4943-a547-c9b3623fe385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:59.668997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1243/2499] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:52:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:52:59.681787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840bba1-84f7-4943-a547-c9b3623fe385 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:52:59.683044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:52:59.685135 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:52:59.687239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:52:59.692197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:59.692197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:59.692197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] PATH_INFO: `/v3/users/4e0e6a33679f4cac96704f71dcfa5c01` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:59.692197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0e6a33679f4cac96704f71dcfa5c01)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:52:59.692197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:52:59.698829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd527ccf-4e0c-4b55-a912-22ee02e69107 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:52:59.704174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:52:59.704776 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:52:59.705224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:52:59.705343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:52:59.714029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:52:59.715660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:52:59.727199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1244/2500] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:52:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:52:59.751147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0e6a33679f4cac96704f71dcfa5c01'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:52:59.759466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0e6a33679f4cac96704f71dcfa5c01'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:52:59.760131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0e6a33679f4cac96704f71dcfa5c01'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:52:59.760761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0e6a33679f4cac96704f71dcfa5c01'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:52:59.768446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7b76a774-e5c6-4874-b737-769743b547db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0e6a33679f4cac96704f71dcfa5c01'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:52:59.785518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1257/2501] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:52:59 2022] DELETE /identity/v3/users/4e0e6a33679f4cac96704f71dcfa5c01 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:52:59.957635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:52:59.968808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89be8-15af-44be-a135-935de15b4bee service placement] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.025397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:00.033318 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.033964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.034223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.034496 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] PATH_INFO: `/v3/projects/7090f91f78534107ba4b5ccd7f80e145` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.035708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7090f91f78534107ba4b5ccd7f80e145)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.037672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.207994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7090f91f78534107ba4b5ccd7f80e145', 'request_id': 'req-cc812968-6469-43ac-82ce-9f40fd1eced4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:53:00.213578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7090f91f78534107ba4b5ccd7f80e145', 'request_id': 'req-cc812968-6469-43ac-82ce-9f40fd1eced4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:53:00.222246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7090f91f78534107ba4b5ccd7f80e145', 'request_id': 'req-cc812968-6469-43ac-82ce-9f40fd1eced4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:53:00.235317 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] The token cache is being invalidate because project 7090f91f78534107ba4b5ccd7f80e145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:53:00.235558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:53:00.236194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cc812968-6469-43ac-82ce-9f40fd1eced4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:53:00.237673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1245/2502] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:52:59 2022] DELETE /identity/v3/projects/7090f91f78534107ba4b5ccd7f80e145 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:53:00.242963 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.243248 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.243553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.289896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:00.291080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:00.348383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1258/2503] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:00.356782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.357064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.357336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.376414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:00.377610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:00.385357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1246/2504] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:00.389433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.398806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce821ed9-2c8d-4d87-80a6-3125bd9b6841 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.403891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.404474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.404752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.405022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.405683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:53:00.405979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.407584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.420514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1259/2505] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:53:00.424592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.433592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7196ae4-0eb4-43ef-8bdb-8d47c7b9ad01 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.438163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.438739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.439012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.439276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.439933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.441794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.467970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1247/2506] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:53:00 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:53:00.472089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.482340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b2369-a927-46e8-adb1-1c1f9f0418ca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.515220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:00.538723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.539321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.539635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.539907 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.540537 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.542439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.564028 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1260/2507] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:53:00 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:53:00.568205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.579405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d009c2c-dd89-42c1-8314-f3c2c7990faa admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.586450 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.587045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.587445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.587936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.588584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.590758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.602304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1248/2508] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:53:00.607124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.617190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c18d9b-1393-41a8-acc0-b2f28b0298a5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.622383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aba54-d8fc-40d3-b45a-28244d353b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.622974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aba54-d8fc-40d3-b45a-28244d353b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.623251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aba54-d8fc-40d3-b45a-28244d353b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.623552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aba54-d8fc-40d3-b45a-28244d353b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.624192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aba54-d8fc-40d3-b45a-28244d353b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.626065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aba54-d8fc-40d3-b45a-28244d353b7f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.636597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1261/2509] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:53:00.640845 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.650803 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8206dda-7530-4f36-b572-0ad7757a8173 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.655872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.656511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.656780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.657043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] PATH_INFO: `/v3/projects/3ef4db17e5d245cb9caab70d557f41ae/users/778fc9ee164e46a9b874dd0e20d38392/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.671988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef4db17e5d245cb9caab70d557f41ae, user_id=778fc9ee164e46a9b874dd0e20d38392, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.674639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.688077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1249/2510] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:53:00 2022] PUT /identity/v3/projects/3ef4db17e5d245cb9caab70d557f41ae/users/778fc9ee164e46a9b874dd0e20d38392/roles/87f1831257824560813f954348534540 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:53:00.693180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2626a57-8cf3-459c-8b04-4ff6fa10385a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.693466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2626a57-8cf3-459c-8b04-4ff6fa10385a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.693739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2626a57-8cf3-459c-8b04-4ff6fa10385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.712177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d2626a57-8cf3-459c-8b04-4ff6fa10385a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:00.713365 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d2626a57-8cf3-459c-8b04-4ff6fa10385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:00.770102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1262/2511] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:00.780116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:00.781809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:00.848811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d15a1-4b7b-4886-ba8e-344bd1503bad admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:00.852466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:00.853078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:00.853380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:00.853646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:00.862414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:00.863850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:00.873842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1250/2512] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:53:05.346868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ac7f9-9c52-43c1-9f8a-9586fcb3ba96 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:05.347433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ac7f9-9c52-43c1-9f8a-9586fcb3ba96 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:05.347433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ac7f9-9c52-43c1-9f8a-9586fcb3ba96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:05.390875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ad4ac7f9-9c52-43c1-9f8a-9586fcb3ba96 None None] MFA Rules not processed for user `778fc9ee164e46a9b874dd0e20d38392`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:05.392513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ad4ac7f9-9c52-43c1-9f8a-9586fcb3ba96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:05.462091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1263/2513] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:05.476861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:05.478253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:05.520908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:05.630693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1ccf6-c209-4797-954e-269fae20c6af service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:05.636638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:05.637263 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:05.637638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:05.637812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:05.659617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:05.661454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:05.676230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1251/2514] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:53:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:53:07.280577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00321ab1-55e5-41de-81c9-4d402adce723 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:07.281008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00321ab1-55e5-41de-81c9-4d402adce723 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:07.281123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00321ab1-55e5-41de-81c9-4d402adce723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:07.303131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-00321ab1-55e5-41de-81c9-4d402adce723 None None] MFA Rules not processed for user `778fc9ee164e46a9b874dd0e20d38392`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:07.304371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-00321ab1-55e5-41de-81c9-4d402adce723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:07.312348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1264/2515] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:08.915682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:08.918077 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:08.938427 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930488e2-bf01-4cfd-86ae-89184d8488d3 service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:08.947837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:08.949379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:08.949877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:08.950339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:08.953132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:09.046359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:09.048404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31884891-ee81-4ecc-b232-e746d3202113 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:09.061844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1252/2516] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:53:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:53:09.565088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e9cfe-5b2c-4251-a683-d2dc51703a5b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:09.565728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e9cfe-5b2c-4251-a683-d2dc51703a5b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:09.566206 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e9cfe-5b2c-4251-a683-d2dc51703a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:09.587915 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1f8e9cfe-5b2c-4251-a683-d2dc51703a5b None None] MFA Rules not processed for user `c5307118357e44ffbff5074ae4a94c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:09.589055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1f8e9cfe-5b2c-4251-a683-d2dc51703a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:09.650920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:09.689487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1265/2517] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:19.054666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=68 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:19.681850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:20.497474 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114f113-4c06-4493-848f-659db7968d18 None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:20.498048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114f113-4c06-4493-848f-659db7968d18 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:20.498048 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114f113-4c06-4493-848f-659db7968d18 None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:20.499386 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1253/2518] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:53:20.505910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d40c14-b79e-4634-904b-67bc568162cc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:20.506282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d40c14-b79e-4634-904b-67bc568162cc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:20.506624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d40c14-b79e-4634-904b-67bc568162cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:20.526998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-10d40c14-b79e-4634-904b-67bc568162cc None None] MFA Rules not processed for user `24bf5028fe8d4a619897cf34ba3ff809`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:20.528372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-10d40c14-b79e-4634-904b-67bc568162cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:20.534838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1266/2519] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:53:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:20.549866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114f113-4c06-4493-848f-659db7968d18 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:20.551989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3114f113-4c06-4493-848f-659db7968d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:20.565815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114f113-4c06-4493-848f-659db7968d18 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:20.569667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd6916-dc85-476b-a3c5-df2b990018d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:20.570379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6916-dc85-476b-a3c5-df2b990018d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:20.570723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6916-dc85-476b-a3c5-df2b990018d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:20.571064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6916-dc85-476b-a3c5-df2b990018d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:20.578405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd6916-dc85-476b-a3c5-df2b990018d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:20.579865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd6916-dc85-476b-a3c5-df2b990018d0 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:20.589207 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1254/2520] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:53:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:53:20.693152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a28d3-44cd-4efc-82a6-d0879090d90f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:20.693580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a28d3-44cd-4efc-82a6-d0879090d90f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:20.693688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a28d3-44cd-4efc-82a6-d0879090d90f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:20.694957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1267/2521] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 00:53:20.700661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d65a38-5de2-420e-8bac-edabc4c7d72c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:20.700943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d65a38-5de2-420e-8bac-edabc4c7d72c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:20.701214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d65a38-5de2-420e-8bac-edabc4c7d72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:20.720676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-31d65a38-5de2-420e-8bac-edabc4c7d72c None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:20.721858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-31d65a38-5de2-420e-8bac-edabc4c7d72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:20.730828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1255/2522] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:53:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:20.741256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a28d3-44cd-4efc-82a6-d0879090d90f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:20.752647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a28d3-44cd-4efc-82a6-d0879090d90f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:20.756712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0d370-9c9c-4076-8f12-14214e627642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:20.757297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0d370-9c9c-4076-8f12-14214e627642 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:20.757569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0d370-9c9c-4076-8f12-14214e627642 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:20.757835 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0d370-9c9c-4076-8f12-14214e627642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:20.768385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0d370-9c9c-4076-8f12-14214e627642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:20.769796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0d370-9c9c-4076-8f12-14214e627642 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:20.781443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1268/2523] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:53:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:53:30.715926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=9 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:30.776274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=9 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:32.600761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb6a9-735c-4ae0-a28b-fde20a48edd8 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:32.601296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb6a9-735c-4ae0-a28b-fde20a48edd8 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:32.601420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb6a9-735c-4ae0-a28b-fde20a48edd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:32.622149 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b70cb6a9-735c-4ae0-a28b-fde20a48edd8 None None] MFA Rules not processed for user `36e8e45674624c219b59abb39d8dcf83`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:32.623349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b70cb6a9-735c-4ae0-a28b-fde20a48edd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:32.689611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1256/2524] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:37.311242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:37.311746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:37.311848 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:37.330481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] MFA Rules not processed for user `d308262224af475091e6aa494ea839b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:37.331733 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:37.387120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1269/2525] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:42.682094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=25 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:47.380439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=25 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:53:58.386844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:58.387391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:58.387391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:58.406738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:53:58.408102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:58.464437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1257/2526] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 00:53:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:53:58.513949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:53:58.515353 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:53:58.592661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b363e0-df6c-41d4-ad00-620df7904cb2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:53:58.597738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:53:58.598340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:53:58.598616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:53:58.598885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:53:58.607004 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:53:58.608640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:53:58.616829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1270/2527] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:53:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:54:05.226822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:54:05.228383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:05.243368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156c1b-20c7-4d97-b1ff-fcf2ceb4a37f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:54:05.248004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:54:05.248612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:54:05.249128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:54:05.249393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] PATH_INFO: `/v3/users/5d47be4ab52f4034ac09fbf043990709` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:54:05.250539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d47be4ab52f4034ac09fbf043990709)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:54:05.252493 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:54:05.297557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d47be4ab52f4034ac09fbf043990709'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.304744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d47be4ab52f4034ac09fbf043990709'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.305423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d47be4ab52f4034ac09fbf043990709'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.313026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d47be4ab52f4034ac09fbf043990709'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.313702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d47be4ab52f4034ac09fbf043990709'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.331186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1258/2528] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:54:05 2022] DELETE /identity/v3/users/5d47be4ab52f4034ac09fbf043990709 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:54:05.501553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:54:05.512698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae868b-3178-4bfe-824e-31e94c82de88 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:54:05.575629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:54:05.576258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:54:05.576534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:54:05.576802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] PATH_INFO: `/v3/projects/762beb87caad4aa691528a7d9de01086` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:54:05.577964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762beb87caad4aa691528a7d9de01086)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:54:05.579902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:54:05.625618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762beb87caad4aa691528a7d9de01086', 'request_id': 'req-bbe4de8f-0f68-45be-940b-000387704115'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.631226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762beb87caad4aa691528a7d9de01086', 'request_id': 'req-bbe4de8f-0f68-45be-940b-000387704115'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.639178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762beb87caad4aa691528a7d9de01086', 'request_id': 'req-bbe4de8f-0f68-45be-940b-000387704115'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.650108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] The token cache is being invalidate because project 762beb87caad4aa691528a7d9de01086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:54:05.650410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.651047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.652545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1271/2529] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:54:05 2022] DELETE /identity/v3/projects/762beb87caad4aa691528a7d9de01086 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:54:05.656466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:54:05.657847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:05.673050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ea75c-ff8f-4a91-80d0-f323f83f54af admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:54:05.731138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:54:05.731709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:54:05.731992 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:54:05.732746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] PATH_INFO: `/v3/users/b2a3d16e2c01431e857eaead6bd70b07` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:54:05.733865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a3d16e2c01431e857eaead6bd70b07)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:54:05.735798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:54:05.756814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:54:05.782784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a3d16e2c01431e857eaead6bd70b07'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.789349 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a3d16e2c01431e857eaead6bd70b07'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.790027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a3d16e2c01431e857eaead6bd70b07'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.797221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a3d16e2c01431e857eaead6bd70b07'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.797843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a3d16e2c01431e857eaead6bd70b07'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:05.814824 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1259/2530] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:54:05 2022] DELETE /identity/v3/users/b2a3d16e2c01431e857eaead6bd70b07 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:54:05.993623 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:54:05.995748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:06.010679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4de8f-0f68-45be-940b-000387704115 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:54:06.030689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:54:06.075511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:54:06.076190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:54:06.076464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:54:06.076758 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] PATH_INFO: `/v3/projects/0293f640bfc245fa9f271bfcd15a9036` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:54:06.077858 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0293f640bfc245fa9f271bfcd15a9036)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:54:06.079818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:54:06.125607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0293f640bfc245fa9f271bfcd15a9036', 'request_id': 'req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:06.130997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0293f640bfc245fa9f271bfcd15a9036', 'request_id': 'req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:06.138316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0293f640bfc245fa9f271bfcd15a9036', 'request_id': 'req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:06.149135 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] The token cache is being invalidate because project 0293f640bfc245fa9f271bfcd15a9036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:54:06.149417 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:06.150035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:54:06.152060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1272/2531] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:54:05 2022] DELETE /identity/v3/projects/0293f640bfc245fa9f271bfcd15a9036 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:54:15.811489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:54:16.146077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:54:56.265696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:54:56.267109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:56.352046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505993eb-57dc-43c2-aacc-32508efb34fe admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:54:56.397976 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:54:56.398561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:54:56.398835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:54:56.399099 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:54:56.400837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:56.468406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:54:56.469805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0872597-2e73-4359-a6fa-5367c81df395 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:54:56.480463 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1260/2532] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:54:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:54:57.004921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:54:57.006266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:57.022454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5e3c1-ed2e-4a54-a455-03edf7b98f2b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:54:57.027860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:54:57.028451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:54:57.028728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:54:57.028999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:54:57.030665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:54:57.108598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:54:57.110194 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:54:57.128803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1273/2533] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:54:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:05.445029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:05.445468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:05.445576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:05.464617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] MFA Rules not processed for user `778fc9ee164e46a9b874dd0e20d38392`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:05.465797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:05.497430 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:05.520497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1261/2534] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:06.146769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:06.148162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:06.157485 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:06.158928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:06.223511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e6dbb-4fad-43cc-842e-52b0ebc0d26a service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:06.223694 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4254b7-175b-449f-9282-af3819531e9a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:06.228020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:06.228826 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:06.228959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:06.229177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:06.229438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:06.229550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:06.229808 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:06.230091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:06.231199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:06.233513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:06.245405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:06.245676 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:06.246816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:06.247252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:06.257875 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1274/2535] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:06.258181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1262/2536] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:55:08.285400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:08.295506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137a0b2-b63c-4d24-b341-c0b0cd29f7f4 service placement] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:08.299773 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:08.300449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:08.300790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:08.301150 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:08.302940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:08.315055 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:08.316643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:08.327051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1275/2537] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:16.253074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=34 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:18.321701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=79 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:21.731828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:21.733214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:21.746917 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bbc20-d56a-4e48-ae67-337e29763b94 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:21.754981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:21.755623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:21.755905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:21.756176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:21.757972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:21.774962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:21.776604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:21.788348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1263/2538] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:28.364728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:28.366320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:28.381901 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8585a0a-530a-40f3-8234-cddc5cd5333d service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:28.385741 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:28.386347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:28.386617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:28.386891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:28.388704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:28.454608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:28.456266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:28.465976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1276/2539] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:31.783721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=6 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:35.619418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:35.622039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:35.636921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7520a4-d8cb-451c-8e20-23f43111cd28 service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:35.641607 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:35.642224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:35.642503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:35.642778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] PATH_INFO: `/v3/users/65f9c7fc74c448829608dab23d7023ce` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:35.643937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f9c7fc74c448829608dab23d7023ce)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:35.645844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:35.697528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f9c7fc74c448829608dab23d7023ce'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:35.704465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f9c7fc74c448829608dab23d7023ce'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:35.705221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f9c7fc74c448829608dab23d7023ce'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:35.712491 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f9c7fc74c448829608dab23d7023ce'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:35.713125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f9c7fc74c448829608dab23d7023ce'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:35.730342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1264/2540] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:35 2022] DELETE /identity/v3/users/65f9c7fc74c448829608dab23d7023ce => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:35.919264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:35.931972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce462b-bfdd-4cfc-add3-30dad194fbe9 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:35.991034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:35.991681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:35.991958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:35.992226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] PATH_INFO: `/v3/projects/743501da05424df197d7af03d32eabb3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:35.993328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743501da05424df197d7af03d32eabb3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:35.995238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:36.048592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743501da05424df197d7af03d32eabb3', 'request_id': 'req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.054042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743501da05424df197d7af03d32eabb3', 'request_id': 'req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.061902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743501da05424df197d7af03d32eabb3', 'request_id': 'req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.074605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] The token cache is being invalidate because project 743501da05424df197d7af03d32eabb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:36.074894 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.075593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.077087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1277/2541] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:35 2022] DELETE /identity/v3/projects/743501da05424df197d7af03d32eabb3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:36.081104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:36.082512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:36.096120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6bec6-a588-4376-82b0-87e8472f23ab admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:36.156943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:36.158088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:36.158365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:36.158636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] PATH_INFO: `/v3/users/705d7b5148ac4a81a5ad8a6c071f38ad` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:36.159811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=705d7b5148ac4a81a5ad8a6c071f38ad)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:36.161713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:36.207136 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '705d7b5148ac4a81a5ad8a6c071f38ad'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.215068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '705d7b5148ac4a81a5ad8a6c071f38ad'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.215795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '705d7b5148ac4a81a5ad8a6c071f38ad'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.223553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '705d7b5148ac4a81a5ad8a6c071f38ad'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.224217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '705d7b5148ac4a81a5ad8a6c071f38ad'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.232162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:36.244639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1265/2542] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:36 2022] DELETE /identity/v3/users/705d7b5148ac4a81a5ad8a6c071f38ad => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:36.421098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:36.422633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:36.442224 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971a9cb-a6f3-4cd4-af92-bb1a73da1d56 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:36.462161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:36.500820 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:36.502079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:36.502357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:36.502627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] PATH_INFO: `/v3/projects/2fdc30ed86ab4859ad1987f99cf009f0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:36.503790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdc30ed86ab4859ad1987f99cf009f0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:36.505742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:36.550787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdc30ed86ab4859ad1987f99cf009f0', 'request_id': 'req-672946de-8852-4a9d-a563-7873e76222d0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.556256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdc30ed86ab4859ad1987f99cf009f0', 'request_id': 'req-672946de-8852-4a9d-a563-7873e76222d0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.564538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdc30ed86ab4859ad1987f99cf009f0', 'request_id': 'req-672946de-8852-4a9d-a563-7873e76222d0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.576017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] The token cache is being invalidate because project 2fdc30ed86ab4859ad1987f99cf009f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:36.576308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.576985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:36.578506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1278/2543] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:36 2022] DELETE /identity/v3/projects/2fdc30ed86ab4859ad1987f99cf009f0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:38.250764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:38.252933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:38.322782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671df80-fd04-415d-b755-ec7aec655e1a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:38.373003 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:38.373601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:38.373874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:38.374139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:38.376700 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:38.446151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:38.447733 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:38.457981 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1266/2544] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:40.041810 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:40.043420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:40.146289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672946de-8852-4a9d-a563-7873e76222d0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:40.152034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:40.152748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:40.153738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:40.154134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:40.156032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:40.168500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:40.170688 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:40.183510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1279/2545] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:41.780172 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:41.781641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:41.795095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b29bc-f2b2-4987-864e-ca6cbf2848bf service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:41.800020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:41.800641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:41.800926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:41.801192 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] PATH_INFO: `/v3/users/c5307118357e44ffbff5074ae4a94c9d` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:41.802310 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5307118357e44ffbff5074ae4a94c9d)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:41.804244 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:41.851129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5307118357e44ffbff5074ae4a94c9d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:41.854721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:41.859610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5307118357e44ffbff5074ae4a94c9d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:41.860936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5307118357e44ffbff5074ae4a94c9d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:41.869153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5307118357e44ffbff5074ae4a94c9d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:41.869779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e6a1df96-713c-4ff7-9902-2eaafca9075b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5307118357e44ffbff5074ae4a94c9d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:41.887346 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1267/2546] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:41 2022] DELETE /identity/v3/users/c5307118357e44ffbff5074ae4a94c9d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:42.057163 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.078334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e355e2-5481-4be2-8b5c-72bf8b957fa8 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.127016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:42.152887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.153524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.153860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.154138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] PATH_INFO: `/v3/projects/6a9d9cf3606845ec964dd8320126293e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.155296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a9d9cf3606845ec964dd8320126293e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.158106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.209296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a9d9cf3606845ec964dd8320126293e', 'request_id': 'req-7e140b84-90f1-4b09-bb8f-0157f211d39d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:42.215552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a9d9cf3606845ec964dd8320126293e', 'request_id': 'req-7e140b84-90f1-4b09-bb8f-0157f211d39d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:42.224361 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a9d9cf3606845ec964dd8320126293e', 'request_id': 'req-7e140b84-90f1-4b09-bb8f-0157f211d39d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:42.235628 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] The token cache is being invalidate because project 6a9d9cf3606845ec964dd8320126293e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:42.235935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:42.236566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7e140b84-90f1-4b09-bb8f-0157f211d39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:42.238068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1280/2547] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:42 2022] DELETE /identity/v3/projects/6a9d9cf3606845ec964dd8320126293e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:55:42.242994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.243282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.243591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.263105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:42.264385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:42.324600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1268/2548] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:42.332775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.333060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.333344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.354242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:42.355505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:42.365168 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1281/2549] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:42.369419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.380027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef6d2-9d7f-4164-9fb5-5e305262b0d9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.384951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.385576 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.385985 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.386280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.387691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:55:42.388014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.389680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.406702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1269/2550] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:55:42.408449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.417872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fa54d-129e-4d5e-8b65-610db4a116d9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.423022 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f7c42-07dd-47dd-9013-43aa8fc318e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.423739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f7c42-07dd-47dd-9013-43aa8fc318e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.424088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f7c42-07dd-47dd-9013-43aa8fc318e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.424430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f7c42-07dd-47dd-9013-43aa8fc318e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.425162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f7c42-07dd-47dd-9013-43aa8fc318e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.427374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f7c42-07dd-47dd-9013-43aa8fc318e0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.476326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1282/2551] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:55:42.480839 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.486909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.487259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.487571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.493644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8cdc-37b8-4fbc-991f-7169d48e9b63 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.542425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:42.543726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:42.565549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.566130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.566405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.566671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.567293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.569217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.591583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1270/2552] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:55:42.595553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.604950 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88b15-3c3c-41ec-b114-c5c4ee2b2210 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.609650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.610221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.610777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.610777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.611375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.612967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.615593 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1283/2553] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:42.624447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1271/2554] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:55:42.629318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.638363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:42.644152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a10820-61e5-469e-8ab4-598724859ec0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.648943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.649573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.649929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.650653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.651705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.653551 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.666253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.668113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:42.668333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1284/2555] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:42.671891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.681917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d16f7a-9293-4770-a39d-0f7ac40db367 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.686548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd93621-3386-4474-b805-4a8b426e3c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.687189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd93621-3386-4474-b805-4a8b426e3c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.687469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd93621-3386-4474-b805-4a8b426e3c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.687770 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd93621-3386-4474-b805-4a8b426e3c69 admin admin] PATH_INFO: `/v3/projects/a5fa147a4a194ed59a1c40fa8155d196/users/7b0edc75e08046e7bb1facd27002a22f/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.702827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd93621-3386-4474-b805-4a8b426e3c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fa147a4a194ed59a1c40fa8155d196, user_id=7b0edc75e08046e7bb1facd27002a22f, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.706063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd93621-3386-4474-b805-4a8b426e3c69 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.711763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:42.720732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1285/2556] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:42 2022] PUT /identity/v3/projects/a5fa147a4a194ed59a1c40fa8155d196/users/7b0edc75e08046e7bb1facd27002a22f/roles/87f1831257824560813f954348534540 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:55:42.730574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c81d-4918-422d-a60b-dead447632fa None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.730854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c81d-4918-422d-a60b-dead447632fa None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.731131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c81d-4918-422d-a60b-dead447632fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.758589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0797c81d-4918-422d-a60b-dead447632fa None None] MFA Rules not processed for user `7b0edc75e08046e7bb1facd27002a22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:42.759785 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0797c81d-4918-422d-a60b-dead447632fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:42.768313 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae58c1-2eeb-4161-b367-ec5a54b01ac2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.832939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1286/2557] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:55:42.837411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c81d-4918-422d-a60b-dead447632fa None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.840204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.840865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.841189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.841411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.911753 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c81d-4918-422d-a60b-dead447632fa None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:42.918160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:42.918842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:42.919124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:42.919396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:42.920045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.920426 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:42.922091 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.922285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:42.931879 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1272/2558] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 00:55:42.968109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1287/2559] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:55:42.969293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:42.981625 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f6646-b510-4706-8d15-625aeb0d7c14 service swift] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.049039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.049570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.049844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.050110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.050741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.052688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.061206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:43.077184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1273/2560] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:55:42 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:55:43.081703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:43.092916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb02e2e-c731-470b-b867-e0caa53c91c8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.097804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.098398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.098672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.098942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.099786 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.101263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.112498 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1288/2561] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:43.117081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:43.130033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05724c6b-6174-44b0-b7b6-83c7a198be40 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.135734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.136818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.137169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.137605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] PATH_INFO: `/v3/projects/f8e03654c6b0461aaa55a6d56fe424f7/users/552688b334db4e858521b6ef0bcc7a00/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.163869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e03654c6b0461aaa55a6d56fe424f7, user_id=552688b334db4e858521b6ef0bcc7a00, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.166783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.182414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1274/2562] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:43 2022] PUT /identity/v3/projects/f8e03654c6b0461aaa55a6d56fe424f7/users/552688b334db4e858521b6ef0bcc7a00/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:43.186279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:43.197395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f3b02-9caa-41cf-a4d8-e26e7f982737 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.239654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:43.274656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeca08-67db-4f87-a5d7-26c99d5a1f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.275636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeca08-67db-4f87-a5d7-26c99d5a1f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.275915 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeca08-67db-4f87-a5d7-26c99d5a1f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.276178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeca08-67db-4f87-a5d7-26c99d5a1f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.277182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeca08-67db-4f87-a5d7-26c99d5a1f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.279167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeca08-67db-4f87-a5d7-26c99d5a1f7b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.289825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1289/2563] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:43.294022 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:43.305723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e6707-b894-4d4e-8748-819f65c618f4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.310462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.311108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.311377 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.311671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] PATH_INFO: `/v3/projects/f8e03654c6b0461aaa55a6d56fe424f7/users/552688b334db4e858521b6ef0bcc7a00/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.314160 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e03654c6b0461aaa55a6d56fe424f7, user_id=552688b334db4e858521b6ef0bcc7a00, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.316830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.328829 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1275/2564] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:43 2022] PUT /identity/v3/projects/f8e03654c6b0461aaa55a6d56fe424f7/users/552688b334db4e858521b6ef0bcc7a00/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:43.340059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.340341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.340614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.370695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] MFA Rules not processed for user `552688b334db4e858521b6ef0bcc7a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:43.372025 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:43.452550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1290/2565] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:43.464595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:43.466327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:43.532423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d775b6-97bb-4525-ad34-605de19ea859 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.536259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.536895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.537198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.537465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.548158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.550074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.568338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1276/2566] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:43.661421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:43.676473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dfa09-ed60-4de9-8d25-a77f9ff597dd None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:43.682482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01b13a-c798-45c6-b77b-7899a2aaecca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:43.683094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b13a-c798-45c6-b77b-7899a2aaecca service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:43.683366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b13a-c798-45c6-b77b-7899a2aaecca service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:43.683682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b13a-c798-45c6-b77b-7899a2aaecca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:43.758612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b13a-c798-45c6-b77b-7899a2aaecca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:43.760769 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b13a-c798-45c6-b77b-7899a2aaecca service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:43.771416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1291/2567] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:53.559714 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:INSERT=5,keystone:SELECT=43 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:53.767153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=76 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:55.158423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:55.218812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11377b6-3098-49e4-a7d4-7a1ad3b923fc service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:55.223282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:55.223929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:55.224209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:55.224481 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] PATH_INFO: `/v3/users/7b0edc75e08046e7bb1facd27002a22f` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:55.225591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0edc75e08046e7bb1facd27002a22f)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:55.227507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:55.272960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0edc75e08046e7bb1facd27002a22f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.280190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0edc75e08046e7bb1facd27002a22f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.280806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0edc75e08046e7bb1facd27002a22f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.287513 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0edc75e08046e7bb1facd27002a22f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.288142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0edc75e08046e7bb1facd27002a22f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.306045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1277/2568] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:55 2022] DELETE /identity/v3/users/7b0edc75e08046e7bb1facd27002a22f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:55.310493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:55.310955 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:55.311054 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:55.330048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:55.331226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:55.387305 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1292/2569] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:55.397053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:55.398451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:55.455617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1c80-2460-48af-9982-53b105c932b9 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:55.459230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:55.459848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:55.460125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:55.460395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:55.469219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:55.471074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:55.480422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1278/2570] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:55.750407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:55.752234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:55.766017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34405bf-1968-4313-aafa-ebbb0accaed3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:55.771216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:55.772193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:55.772480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:55.772761 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] PATH_INFO: `/v3/projects/a5fa147a4a194ed59a1c40fa8155d196` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:55.773878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fa147a4a194ed59a1c40fa8155d196)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:55.775840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:55.823035 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fa147a4a194ed59a1c40fa8155d196', 'request_id': 'req-00595afe-25a8-4390-9192-cf5a10035093'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.828640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fa147a4a194ed59a1c40fa8155d196', 'request_id': 'req-00595afe-25a8-4390-9192-cf5a10035093'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.836378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fa147a4a194ed59a1c40fa8155d196', 'request_id': 'req-00595afe-25a8-4390-9192-cf5a10035093'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.848294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] The token cache is being invalidate because project a5fa147a4a194ed59a1c40fa8155d196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:55.848590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.849230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.850706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1293/2571] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:55 2022] DELETE /identity/v3/projects/a5fa147a4a194ed59a1c40fa8155d196 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:55.854766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:55.864634 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67027f-542c-47da-a860-5eeed1777634 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:55.882962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:55.923617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:55.924227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:55.924504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:55.924764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] PATH_INFO: `/v3/users/552688b334db4e858521b6ef0bcc7a00` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:55.925869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=552688b334db4e858521b6ef0bcc7a00)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:55.927785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:55.973684 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552688b334db4e858521b6ef0bcc7a00'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.980565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552688b334db4e858521b6ef0bcc7a00'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.981223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552688b334db4e858521b6ef0bcc7a00'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.988013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552688b334db4e858521b6ef0bcc7a00'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:55.988638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a3a830da-daeb-409f-979a-01d5831deba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552688b334db4e858521b6ef0bcc7a00'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:56.005806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1279/2572] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:55 2022] DELETE /identity/v3/users/552688b334db4e858521b6ef0bcc7a00 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:56.317198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.318624 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:56.333512 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00595afe-25a8-4390-9192-cf5a10035093 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.391792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.392421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.392698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.393087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] PATH_INFO: `/v3/projects/f8e03654c6b0461aaa55a6d56fe424f7` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.394071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e03654c6b0461aaa55a6d56fe424f7)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.396037 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.426340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:56.445005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e03654c6b0461aaa55a6d56fe424f7', 'request_id': 'req-0a26731f-f471-45d4-836d-367936734d56'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:56.450435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e03654c6b0461aaa55a6d56fe424f7', 'request_id': 'req-0a26731f-f471-45d4-836d-367936734d56'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:56.458917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e03654c6b0461aaa55a6d56fe424f7', 'request_id': 'req-0a26731f-f471-45d4-836d-367936734d56'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:56.470425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] The token cache is being invalidate because project f8e03654c6b0461aaa55a6d56fe424f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:56.470710 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:56.471340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0a26731f-f471-45d4-836d-367936734d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:56.472891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1294/2573] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:56 2022] DELETE /identity/v3/projects/f8e03654c6b0461aaa55a6d56fe424f7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:56.477827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.478298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.478409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.497643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:56.498843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:56.555887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1280/2574] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:56.563743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.564026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.564298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.585729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:56.587049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:56.595687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1295/2575] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:56.599889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.609351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63565a4-7616-43e2-9ce8-fad533178f09 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.613912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.614502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.614777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.615043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.615721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:55:56.616018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.617594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.629958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1281/2576] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:55:56.634252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.644467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f068d-5789-4ad0-976f-f8e6bf33bee5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.650117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.650771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.651098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.651585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.652345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.655076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.685802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1296/2577] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:55:56 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:55:56.690171 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.701687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2efb74-3535-4f56-a34c-1861c670072a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.728569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:56.760670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.761268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.761544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.761814 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.762445 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.764380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.786225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1282/2578] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:55:56 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:55:56.790352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.800919 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0809-e28f-48ef-801a-1d3d86f38a78 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.805584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.806225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.806455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.806729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.807373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.809017 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.819683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1297/2579] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:56.823840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.834306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f87f78-7cae-40a4-ad3d-694ad4c67339 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.838997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c5d4-1a92-484d-b89b-1218dd48a757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.839680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c5d4-1a92-484d-b89b-1218dd48a757 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.840084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c5d4-1a92-484d-b89b-1218dd48a757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.840357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c5d4-1a92-484d-b89b-1218dd48a757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.841021 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4c5d4-1a92-484d-b89b-1218dd48a757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.842588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4c5d4-1a92-484d-b89b-1218dd48a757 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.852937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1283/2580] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:56.857341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:56.868529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af39a6a-fd41-4a73-a80b-2c97e0ea4afb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:56.873917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:56.874675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.875008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.875318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] PATH_INFO: `/v3/projects/d502850da4e74af6ae2a8477407f4a2c/users/0ceba8bf9acc48f2bbfa68d8d09b8e9a/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.892266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d502850da4e74af6ae2a8477407f4a2c, user_id=0ceba8bf9acc48f2bbfa68d8d09b8e9a, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:56.895258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:56.909072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1298/2581] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:56 2022] PUT /identity/v3/projects/d502850da4e74af6ae2a8477407f4a2c/users/0ceba8bf9acc48f2bbfa68d8d09b8e9a/roles/87f1831257824560813f954348534540 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:56.920627 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:56.921023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:56.921300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:56.948990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] MFA Rules not processed for user `0ceba8bf9acc48f2bbfa68d8d09b8e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:56.950211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:57.013089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1284/2582] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:57.018093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.077780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5193-0500-4736-8674-c8c810e984a2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.083093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.083716 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.084014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.084272 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.084908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.086802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.114876 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1299/2583] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:55:57 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:55:57.118788 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.129787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1352-919b-4977-8365-ff0942cdc570 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.184489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.185197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.185470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.185865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.186489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.189026 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.209470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1285/2584] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:55:57 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:55:57.213478 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.224304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5fbf-f689-4f76-a273-006cd520cca5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.228985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.229611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.229890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.230162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.230803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.232437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.243277 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1300/2585] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:57.247157 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.257659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069266a5-1ca3-4da5-aab3-e83e23dbef85 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.262251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.262889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.263167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.263437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] PATH_INFO: `/v3/projects/20a7ed30cf194beb8eb6dd028dc51934/users/c58d542ba8fd4eb19201295532000ba1/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.277898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a7ed30cf194beb8eb6dd028dc51934, user_id=c58d542ba8fd4eb19201295532000ba1, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.280601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.293666 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1286/2586] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:57 2022] PUT /identity/v3/projects/20a7ed30cf194beb8eb6dd028dc51934/users/c58d542ba8fd4eb19201295532000ba1/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:55:57.298171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.308235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58558a74-5067-4815-9823-f7943586fac3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.330931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:57.369843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26696752-f6b8-4111-a987-5023abd365b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.370462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26696752-f6b8-4111-a987-5023abd365b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.370734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26696752-f6b8-4111-a987-5023abd365b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.371000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26696752-f6b8-4111-a987-5023abd365b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.371668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26696752-f6b8-4111-a987-5023abd365b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.373254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26696752-f6b8-4111-a987-5023abd365b5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.384199 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1301/2587] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:57.388333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.393771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:57.400283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2796-ae67-4d05-a6b7-be26bc97a1d0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.405383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.405968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.406351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.406484 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] PATH_INFO: `/v3/projects/20a7ed30cf194beb8eb6dd028dc51934/users/c58d542ba8fd4eb19201295532000ba1/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.408476 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a7ed30cf194beb8eb6dd028dc51934, user_id=c58d542ba8fd4eb19201295532000ba1, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.411096 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.421832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1287/2588] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:57 2022] PUT /identity/v3/projects/20a7ed30cf194beb8eb6dd028dc51934/users/c58d542ba8fd4eb19201295532000ba1/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:57.433192 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca57844-de53-4f31-8311-e336990d76df None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.433494 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca57844-de53-4f31-8311-e336990d76df None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.433857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca57844-de53-4f31-8311-e336990d76df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.463314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cca57844-de53-4f31-8311-e336990d76df None None] MFA Rules not processed for user `c58d542ba8fd4eb19201295532000ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:57.464631 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cca57844-de53-4f31-8311-e336990d76df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:57.534839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1302/2589] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:57.546615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:57.549044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:57.612670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c505-bd15-4b6b-a7e5-8a7b232caa6f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:57.616308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09dc8a-da0b-435a-a979-fc84cd059fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:57.616908 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09dc8a-da0b-435a-a979-fc84cd059fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:57.617182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09dc8a-da0b-435a-a979-fc84cd059fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:57.617451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09dc8a-da0b-435a-a979-fc84cd059fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:57.628505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09dc8a-da0b-435a-a979-fc84cd059fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:57.629991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09dc8a-da0b-435a-a979-fc84cd059fb2 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:57.640121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1288/2590] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:58.082544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca57844-de53-4f31-8311-e336990d76df None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:58.155679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca57844-de53-4f31-8311-e336990d76df None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:58.161971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:58.162613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.162938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.163245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] PATH_INFO: `/v3/users/0ceba8bf9acc48f2bbfa68d8d09b8e9a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.164703 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ceba8bf9acc48f2bbfa68d8d09b8e9a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:58.167659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:58.218598 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ceba8bf9acc48f2bbfa68d8d09b8e9a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.220878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:58.228137 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ceba8bf9acc48f2bbfa68d8d09b8e9a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.229058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ceba8bf9acc48f2bbfa68d8d09b8e9a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.230059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ceba8bf9acc48f2bbfa68d8d09b8e9a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.237284 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ceba8bf9acc48f2bbfa68d8d09b8e9a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.255715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1303/2591] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:58 2022] DELETE /identity/v3/users/0ceba8bf9acc48f2bbfa68d8d09b8e9a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:58.260307 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935e0ff-73c6-4656-957f-c7bca9fc8d7e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.260590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935e0ff-73c6-4656-957f-c7bca9fc8d7e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.260860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935e0ff-73c6-4656-957f-c7bca9fc8d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.290850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1935e0ff-73c6-4656-957f-c7bca9fc8d7e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:58.292086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1935e0ff-73c6-4656-957f-c7bca9fc8d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:58.354339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1289/2592] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:58.364393 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:58.365722 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:58.436759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fcf6d-5eb9-417e-a9f7-e0256bbde5c3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:58.439141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20822e3-0926-471d-ab97-79c9bfb9198f None None] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.439747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20822e3-0926-471d-ab97-79c9bfb9198f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.440216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20822e3-0926-471d-ab97-79c9bfb9198f None None] PATH_INFO: `/` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.441527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:58.442226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.442575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.442855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1290/2593] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 00:55:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 00:55:58.442977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.447796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.448268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.448715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.453661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:58.455890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:58.467176 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1304/2594] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:55:58.490715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] MFA Rules not processed for user `4784d64225054ea9a987581a00b6807b`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:58.491967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:58.561846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1291/2595] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:55:58.574629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:58.576214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:58.652965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6124f-bd2b-493a-a33e-40867ffe4a28 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:58.657122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:58.657738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.658018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.658291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.669812 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:58.671715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:58.683851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1305/2596] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:55:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:55:58.823230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:58.825034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:58.841589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b59db6-7023-4278-8f1a-7867ce979145 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:58.847851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:58.848934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:58.849280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:58.849622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] PATH_INFO: `/v3/projects/d502850da4e74af6ae2a8477407f4a2c` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:58.850834 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=d502850da4e74af6ae2a8477407f4a2c)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:58.853122 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:58.861279 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:58.904520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd502850da4e74af6ae2a8477407f4a2c', 'request_id': 'req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.910361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd502850da4e74af6ae2a8477407f4a2c', 'request_id': 'req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.918306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd502850da4e74af6ae2a8477407f4a2c', 'request_id': 'req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.930210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] The token cache is being invalidate because project d502850da4e74af6ae2a8477407f4a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:58.930492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.931108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:58.932598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1292/2597] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:58 2022] DELETE /identity/v3/projects/d502850da4e74af6ae2a8477407f4a2c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:58.936520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:58.947667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee708676-cd5c-417a-8382-ce1a7e06281e service glance] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.012015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.012649 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.012950 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.013222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] PATH_INFO: `/v3/users/c58d542ba8fd4eb19201295532000ba1` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.014386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58d542ba8fd4eb19201295532000ba1)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.016420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.030470 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=1,keystone:DELETE=4,keystone:SELECT=95 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:59.066549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58d542ba8fd4eb19201295532000ba1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.073732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58d542ba8fd4eb19201295532000ba1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.074336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58d542ba8fd4eb19201295532000ba1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.075053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58d542ba8fd4eb19201295532000ba1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.082092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7d58141d-21db-4dd1-9851-3ac578e50beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58d542ba8fd4eb19201295532000ba1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.102282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1306/2598] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:55:58 2022] DELETE /identity/v3/users/c58d542ba8fd4eb19201295532000ba1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:59.410637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.412116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:59.425249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f67cd-9b6b-4a16-bd8a-da934cc85dda admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.492037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.492699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.493005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.493278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] PATH_INFO: `/v3/projects/20a7ed30cf194beb8eb6dd028dc51934` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.494425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a7ed30cf194beb8eb6dd028dc51934)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.496433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.543869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a7ed30cf194beb8eb6dd028dc51934', 'request_id': 'req-a088a7d8-f447-4819-8d3a-e1e7c1c30245'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.550553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a7ed30cf194beb8eb6dd028dc51934', 'request_id': 'req-a088a7d8-f447-4819-8d3a-e1e7c1c30245'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.559089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a7ed30cf194beb8eb6dd028dc51934', 'request_id': 'req-a088a7d8-f447-4819-8d3a-e1e7c1c30245'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.572807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] The token cache is being invalidate because project 20a7ed30cf194beb8eb6dd028dc51934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:55:59.573404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.573777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a088a7d8-f447-4819-8d3a-e1e7c1c30245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:55:59.575311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1293/2599] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:55:59 2022] DELETE /identity/v3/projects/20a7ed30cf194beb8eb6dd028dc51934 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:55:59.581202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548f413-4f60-4897-af47-da47be501d79 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.581671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548f413-4f60-4897-af47-da47be501d79 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.581798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548f413-4f60-4897-af47-da47be501d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.601695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c548f413-4f60-4897-af47-da47be501d79 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:59.603254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c548f413-4f60-4897-af47-da47be501d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:59.674290 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1307/2600] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:59.678124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.678429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.678695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.699467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:55:59.700715 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:55:59.709182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1294/2601] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:55:59.713856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548f413-4f60-4897-af47-da47be501d79 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.723781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548f413-4f60-4897-af47-da47be501d79 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.728778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.729378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.729655 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.729920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.730569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:55:59.730922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.732517 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.745326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1308/2602] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:55:59.749597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.759730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7e14-ea83-41d3-a90e-c148ef26adac None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.764434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.765012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.765291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.765748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.766414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.768348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.784166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:59.802380 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1295/2603] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:55:59 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:55:59.806490 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.817363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cb1ae-5dff-4388-b589-630ebc4857f4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.874525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.875134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.875410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.875861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.876742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.878743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.901278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1309/2604] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:55:59 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:55:59.905309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.915929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b173-9d0b-4cbd-b3d9-f41fb95b5c37 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.920567 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.921155 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.921432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.921701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.922342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.924617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.934928 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1296/2605] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:59.939032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.949281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ec3e6-8dc2-487d-9aed-7f0f3830a4ff admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.953952 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3274243-5252-4d62-8b34-daa629cfa8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.954690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274243-5252-4d62-8b34-daa629cfa8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.954973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274243-5252-4d62-8b34-daa629cfa8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.955242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274243-5252-4d62-8b34-daa629cfa8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:55:59.956066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3274243-5252-4d62-8b34-daa629cfa8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:55:59.958247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3274243-5252-4d62-8b34-daa629cfa8dc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:55:59.960591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:55:59.974098 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1310/2606] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:55:59.978671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:55:59.989479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dfea-8759-4861-899f-8dd0eba86be8 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:55:59.994104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:55:59.994752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:55:59.995032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:55:59.995319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] PATH_INFO: `/v3/projects/289f1c9678c447d4852df356b587d293/users/98e5342496b4407e824a793797254221/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.010615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289f1c9678c447d4852df356b587d293, user_id=98e5342496b4407e824a793797254221, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:00.013342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:00.027846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1297/2607] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:55:59 2022] PUT /identity/v3/projects/289f1c9678c447d4852df356b587d293/users/98e5342496b4407e824a793797254221/roles/87f1831257824560813f954348534540 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:00.039818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.040180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.040521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.069112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] MFA Rules not processed for user `98e5342496b4407e824a793797254221`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:00.070330 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.134904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1311/2608] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:00.146002 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:00.147750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.206615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e5996-170f-40ed-9c9b-93f26d708df5 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:00.210532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23a59a-6cc0-4c62-bcaf-1ee2d559c8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:00.211145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a59a-6cc0-4c62-bcaf-1ee2d559c8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.211422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a59a-6cc0-4c62-bcaf-1ee2d559c8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.211725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a59a-6cc0-4c62-bcaf-1ee2d559c8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.221128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23a59a-6cc0-4c62-bcaf-1ee2d559c8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:00.222555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23a59a-6cc0-4c62-bcaf-1ee2d559c8c7 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:00.234355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1298/2609] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:00.256293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:00.319771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e8b1-a482-49de-ba66-0860eda3f4b3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:00.324382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:00.325014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.325296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.325568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] PATH_INFO: `/v3/users/98e5342496b4407e824a793797254221` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.326697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e5342496b4407e824a793797254221)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:00.328658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:00.377126 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e5342496b4407e824a793797254221'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:00.384872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e5342496b4407e824a793797254221'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:00.385612 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e5342496b4407e824a793797254221'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:00.386306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e5342496b4407e824a793797254221'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:00.394324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e5342496b4407e824a793797254221'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:00.414468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1312/2610] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:56:00 2022] DELETE /identity/v3/users/98e5342496b4407e824a793797254221 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:00.418916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30eeb7-fe18-4180-bb4a-cfcd44aedfb1 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.419197 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30eeb7-fe18-4180-bb4a-cfcd44aedfb1 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.419460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30eeb7-fe18-4180-bb4a-cfcd44aedfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.439308 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-be30eeb7-fe18-4180-bb4a-cfcd44aedfb1 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:00.440643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-be30eeb7-fe18-4180-bb4a-cfcd44aedfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.503342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1299/2611] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:00.513080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:00.514464 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.521406 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:00.573409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643c87-c43e-463d-abc5-0662ae2d78ce admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:00.577067 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:00.577666 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.577967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.578239 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.587906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:00.589341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:00.599725 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1313/2612] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:00.682590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.683047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.683341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.706490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:00.707820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.765102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1300/2613] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:00.778252 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:00.779748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.863872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728ce6c-3a32-4192-9671-55f323b6214a service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:00.869859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bdbbc-3685-4e3c-b32b-df5688d49dcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:00.870603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bdbbc-3685-4e3c-b32b-df5688d49dcc service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.870884 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bdbbc-3685-4e3c-b32b-df5688d49dcc service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.871265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bdbbc-3685-4e3c-b32b-df5688d49dcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.881641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bdbbc-3685-4e3c-b32b-df5688d49dcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:00.883259 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bdbbc-3685-4e3c-b32b-df5688d49dcc service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:00.883409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:00.884837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:00.894486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1314/2614] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:56:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:56:00.907724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c663197-5c97-40b5-bdeb-1d571c2713fe None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:00.913620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:00.914299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:00.914578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:00.914853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] PATH_INFO: `/v3/projects/289f1c9678c447d4852df356b587d293` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:00.916132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=289f1c9678c447d4852df356b587d293)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:00.918125 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:00.927933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:01.010524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289f1c9678c447d4852df356b587d293', 'request_id': 'req-d8e9b9be-480d-42dc-83ff-3ef018ed3343'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:01.021188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289f1c9678c447d4852df356b587d293', 'request_id': 'req-d8e9b9be-480d-42dc-83ff-3ef018ed3343'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:01.032861 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289f1c9678c447d4852df356b587d293', 'request_id': 'req-d8e9b9be-480d-42dc-83ff-3ef018ed3343'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:01.062272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] The token cache is being invalidate because project 289f1c9678c447d4852df356b587d293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:56:01.062655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:01.065282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d8e9b9be-480d-42dc-83ff-3ef018ed3343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:01.067517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1301/2615] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:56:00 2022] DELETE /identity/v3/projects/289f1c9678c447d4852df356b587d293 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:56:01.072497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.073114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.073519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.097491 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:01.099133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:01.180376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1315/2616] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:56:01.189679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.189966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.190251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.214376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:01.215820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:01.224298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1302/2617] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:01.229186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.240954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e6786-80be-4732-9bc4-69cf39693ffe None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.246522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.247219 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.247584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.247968 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.248825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:56:01.249187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.251634 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.272522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.279101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1316/2618] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 00:56:01.285327 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28f280-6b22-465d-b640-a90f1a759709 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.291936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.293254 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.293640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.294000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.295143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.302461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.342204 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1303/2619] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:56:01 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:56:01.347276 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.357622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd735491-a727-47e7-bebe-1cb1a942b905 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.384867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:01.426685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.427372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.427680 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.427969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.428645 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.432856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.464357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1317/2620] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:56:01 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:56:01.465644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.477882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746deca-6960-4d01-b644-334486d76cc5 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.483241 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.483909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.484195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.484466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.485736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.487667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.498338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1304/2621] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:56:01.502643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.513190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa019443-73c6-44db-914e-1d1d0564d85c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.517906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09c43a-ce37-44d1-a42f-a5a24467cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.518505 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c43a-ce37-44d1-a42f-a5a24467cda5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.518778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c43a-ce37-44d1-a42f-a5a24467cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.519048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c43a-ce37-44d1-a42f-a5a24467cda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.519742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09c43a-ce37-44d1-a42f-a5a24467cda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.521336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09c43a-ce37-44d1-a42f-a5a24467cda5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.533579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1318/2622] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:56:01.537751 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.548483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e66244-beb4-4f75-85e5-e74c151f2789 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.553341 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.553996 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.554270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.554550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] PATH_INFO: `/v3/projects/2fff7c9ef8ce449cb42581a1cd5f194a/users/1530c2b554b5428cbb874d3046226f32/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.578311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fff7c9ef8ce449cb42581a1cd5f194a, user_id=1530c2b554b5428cbb874d3046226f32, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.581105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.595807 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1305/2623] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:56:01 2022] PUT /identity/v3/projects/2fff7c9ef8ce449cb42581a1cd5f194a/users/1530c2b554b5428cbb874d3046226f32/roles/87f1831257824560813f954348534540 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:01.606125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.606723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.606723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.636324 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] MFA Rules not processed for user `1530c2b554b5428cbb874d3046226f32`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:01.637566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:01.714811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1319/2624] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:01.726798 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.728811 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:01.790683 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc20b60-ed19-46af-9102-5610d9b7ae24 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.795018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.795737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.796107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.796435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.805605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.807237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.816799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1306/2625] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:01.842516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.907903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:01.909374 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:01.922007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee8584-7e97-4ea7-9b98-157eccea9f55 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.925958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.926556 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.926832 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.927104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.927382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002b7f1-abbf-4d1c-97a9-489d1dde4e92 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:01.928916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:01.932106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:01.932815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:01.933146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:01.933481 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] PATH_INFO: `/v3/users/1530c2b554b5428cbb874d3046226f32` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:01.934689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1530c2b554b5428cbb874d3046226f32)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:01.936737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:01.963173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:01.992705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:02.009072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1530c2b554b5428cbb874d3046226f32'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.013444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:02.014888 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c77dc-52bc-4641-a983-26687ceae788 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:02.023687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1530c2b554b5428cbb874d3046226f32'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.024380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1530c2b554b5428cbb874d3046226f32'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.025038 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1530c2b554b5428cbb874d3046226f32'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.027756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1307/2626] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:56:02.049264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1530c2b554b5428cbb874d3046226f32'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.075768 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1320/2627] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:56:01 2022] DELETE /identity/v3/users/1530c2b554b5428cbb874d3046226f32 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:02.077642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.078184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.078579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.100748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:02.102037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:02.221246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1308/2628] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:02.228447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:02.230103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:02.318887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caa4b6-4e43-49d5-9328-cb703c946067 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:02.326850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ef16f-60fd-45c2-a340-b6a330fc6fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:02.327713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ef16f-60fd-45c2-a340-b6a330fc6fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.328072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ef16f-60fd-45c2-a340-b6a330fc6fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.328349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ef16f-60fd-45c2-a340-b6a330fc6fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.338958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ef16f-60fd-45c2-a340-b6a330fc6fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:02.340817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ef16f-60fd-45c2-a340-b6a330fc6fa3 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:02.359155 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1321/2629] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:02.663068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:02.664682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:02.687710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b20fc3-77ed-4be3-a244-2fff8af6ee79 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:02.698275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:02.699052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.699499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.700031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] PATH_INFO: `/v3/projects/2fff7c9ef8ce449cb42581a1cd5f194a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.701315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fff7c9ef8ce449cb42581a1cd5f194a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:02.703345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:02.754966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fff7c9ef8ce449cb42581a1cd5f194a', 'request_id': 'req-bded7d65-54c1-4525-bd0d-933969251862'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.761529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fff7c9ef8ce449cb42581a1cd5f194a', 'request_id': 'req-bded7d65-54c1-4525-bd0d-933969251862'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.769846 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fff7c9ef8ce449cb42581a1cd5f194a', 'request_id': 'req-bded7d65-54c1-4525-bd0d-933969251862'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.781537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] The token cache is being invalidate because project 2fff7c9ef8ce449cb42581a1cd5f194a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:56:02.781826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.783117 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-bded7d65-54c1-4525-bd0d-933969251862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:02.784608 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1309/2630] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:56:02 2022] DELETE /identity/v3/projects/2fff7c9ef8ce449cb42581a1cd5f194a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:02.789450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.789915 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.790013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.811358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:02.812603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:02.878114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1322/2631] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:02.886662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.886936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.887211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.909227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:02.910431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:02.918399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1310/2632] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:02.922698 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:02.934813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edb64e-abb8-4595-b9a8-4931051a7027 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:02.939999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:02.940603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.940877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.941145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.941790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:56:02.942090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:02.943726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:02.957359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1323/2633] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:56:02.961669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:02.971070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558301e1-de61-49ab-85b0-4077defb32fa None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:02.975916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:02.976553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:02.976831 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:02.977108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:02.977756 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:02.979767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:03.011361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1311/2634] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:56:02 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:56:03.015573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:03.029468 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60525c-83a5-48f2-bb5b-a727a5f0b146 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:03.075653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:03.098912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:03.099841 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:03.100220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:03.100493 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:03.101159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:03.103516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:03.135029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1324/2635] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:56:03 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:56:03.139418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:03.150757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45066af-32d4-42ee-8a40-49dc9bd4c7c2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:03.155581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:03.156180 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:03.156455 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:03.156721 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:03.157407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:03.158991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:03.178063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1312/2636] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:56:03.181982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:03.193243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18118e8-c901-4070-a4f0-a88227edeaca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:03.201069 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15e115-0807-48b9-948c-fff94bd27a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:03.202313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15e115-0807-48b9-948c-fff94bd27a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:03.202905 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15e115-0807-48b9-948c-fff94bd27a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:03.204018 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15e115-0807-48b9-948c-fff94bd27a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:03.204986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15e115-0807-48b9-948c-fff94bd27a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:03.206766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15e115-0807-48b9-948c-fff94bd27a05 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:03.220238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1325/2637] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:56:03.224641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:03.241423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee77af4-ab0a-4fd8-981a-1d7a3f7eeeb3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:03.246865 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:03.247587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:03.247858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:03.248777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] PATH_INFO: `/v3/projects/01d7d32119984a1aacc7109e74d9fe01/users/25addc94df284d82b633b3a29954507e/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:03.261154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:03.266411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d7d32119984a1aacc7109e74d9fe01, user_id=25addc94df284d82b633b3a29954507e, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:03.270472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:03.292275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1313/2638] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:56:03 2022] PUT /identity/v3/projects/01d7d32119984a1aacc7109e74d9fe01/users/25addc94df284d82b633b3a29954507e/roles/87f1831257824560813f954348534540 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:03.303139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5701-f6dc-4d59-9b4e-0c582b3d59e6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:03.303743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5701-f6dc-4d59-9b4e-0c582b3d59e6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:03.304397 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5701-f6dc-4d59-9b4e-0c582b3d59e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:03.346431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-100f5701-f6dc-4d59-9b4e-0c582b3d59e6 None None] MFA Rules not processed for user `25addc94df284d82b633b3a29954507e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:03.347948 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-100f5701-f6dc-4d59-9b4e-0c582b3d59e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:03.445906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1326/2639] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:03.457522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:03.459286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:03.544737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761f81-eb03-4d51-a12c-a6b19e3e04e3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:03.549072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60130f4-157f-4ec5-876e-173887f7df89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:03.549800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60130f4-157f-4ec5-876e-173887f7df89 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:03.550372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60130f4-157f-4ec5-876e-173887f7df89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:03.550617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60130f4-157f-4ec5-876e-173887f7df89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:03.560704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60130f4-157f-4ec5-876e-173887f7df89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:03.561780 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60130f4-157f-4ec5-876e-173887f7df89 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:03.571654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1314/2640] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:10.233432 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:10.233986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:10.233986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:10.256010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] MFA Rules not processed for user `25addc94df284d82b633b3a29954507e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:10.257263 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:10.264604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1327/2641] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:13.567237 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:16.379935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7af2e-443f-4558-8a4d-4870e8ac9d1e None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:16.380504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7af2e-443f-4558-8a4d-4870e8ac9d1e None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:16.380504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7af2e-443f-4558-8a4d-4870e8ac9d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:16.399278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-edb7af2e-443f-4558-8a4d-4870e8ac9d1e None None] MFA Rules not processed for user `d308262224af475091e6aa494ea839b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:16.400510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-edb7af2e-443f-4558-8a4d-4870e8ac9d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:16.457507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1315/2642] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:20.249928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:21.575719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:21.578723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:21.673334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df1048-4262-4063-aef2-99e42e177790 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:21.679759 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:21.680383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:21.680667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:21.680940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:21.682829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:21.698854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:21.700842 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:21.714082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1328/2643] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:26.446995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=25 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:26.747550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86eb0e-a543-47ff-b326-b24a3174005f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:26.747833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86eb0e-a543-47ff-b326-b24a3174005f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:26.748100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86eb0e-a543-47ff-b326-b24a3174005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:26.767954 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-be86eb0e-a543-47ff-b326-b24a3174005f None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:26.769132 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-be86eb0e-a543-47ff-b326-b24a3174005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:26.824823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1316/2644] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:26.837614 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:26.839092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:26.913222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e58cf-c322-4d29-aca1-5ae7bec22f2f service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:26.918679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:26.919286 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:26.919592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:26.920449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:26.931383 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:26.933010 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:26.941473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1329/2645] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:56:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:56:36.633305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd2db-70cc-4d27-a9f4-6d759c8390fc None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:36.633772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd2db-70cc-4d27-a9f4-6d759c8390fc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:36.633869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd2db-70cc-4d27-a9f4-6d759c8390fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:36.654392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-c81bd2db-70cc-4d27-a9f4-6d759c8390fc None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:36.655628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-c81bd2db-70cc-4d27-a9f4-6d759c8390fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:36.662822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1317/2646] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:56:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:36.674726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:36.685511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2572d8-9518-4a8f-98c6-5f3175a9142f service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:36.694658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:36.695245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:36.695548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:36.695819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:36.704247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:36.705888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:36.714425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1330/2647] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:56:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:56:37.232103 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b89de-2881-447d-b06f-436b68bca021 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:37.232393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b89de-2881-447d-b06f-436b68bca021 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:37.232670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b89de-2881-447d-b06f-436b68bca021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:37.252919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-279b89de-2881-447d-b06f-436b68bca021 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:37.254070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-279b89de-2881-447d-b06f-436b68bca021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:37.261383 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1318/2648] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:37.273667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:37.285106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c251b-9191-4180-8a6a-8f0566b11712 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:37.290745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:37.291326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:37.291639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:37.291910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:37.299992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:37.301581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:37.310416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1331/2649] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:56:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:56:37.535857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88aa10-9961-4c64-b8db-e1d66d74f50b None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:37.536145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88aa10-9961-4c64-b8db-e1d66d74f50b None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:37.536422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88aa10-9961-4c64-b8db-e1d66d74f50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:37.556428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2b88aa10-9961-4c64-b8db-e1d66d74f50b None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:37.557622 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2b88aa10-9961-4c64-b8db-e1d66d74f50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:37.564748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1319/2650] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:37.576433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:37.587233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb667b38-cd17-46e7-bbf4-94a97159c4a7 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:37.592449 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:37.593059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:37.593333 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:37.593604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:37.602123 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:37.603708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:37.611546 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1332/2651] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:56:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:56:37.921309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:37.921792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:37.921890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:37.940558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] MFA Rules not processed for user `5287027e82ff49e78cae60565bdcc611`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:37.941779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:37.949466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1320/2652] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:37.961828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:37.972097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645af65-b997-4426-8236-7ea3c8a029e8 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:37.976984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c2de9-9535-491e-941c-f659063e850d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0587d0fb93ad478fa931141ac9e1a5de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:37.977563 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c2de9-9535-491e-941c-f659063e850d service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:37.977833 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c2de9-9535-491e-941c-f659063e850d service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:37.978102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c2de9-9535-491e-941c-f659063e850d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:37.985817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c2de9-9535-491e-941c-f659063e850d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:37.987367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c2de9-9535-491e-941c-f659063e850d service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:37.995013 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1333/2653] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 00:56:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 00:56:40.101825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:40.164113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79664c1b-cc5d-4b31-a65a-77bbbcc89938 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:40.168664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:40.169283 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.169557 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.169823 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] PATH_INFO: `/v3/users/25addc94df284d82b633b3a29954507e` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.170941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25addc94df284d82b633b3a29954507e)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:40.172897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:40.222635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25addc94df284d82b633b3a29954507e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.232490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25addc94df284d82b633b3a29954507e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.233512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25addc94df284d82b633b3a29954507e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.241348 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25addc94df284d82b633b3a29954507e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.241973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25addc94df284d82b633b3a29954507e'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.257896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:40.264052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1321/2654] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:56:40 2022] DELETE /identity/v3/users/25addc94df284d82b633b3a29954507e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:40.268890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.269376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.269504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.288699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:40.289817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:40.305965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:40.347746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1334/2655] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:40.357467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:40.358854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:40.414305 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512229ea-6bf0-43ec-a7ba-dcdfee70b9e7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:40.417849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7396d-081b-4177-991d-1916756b2948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:40.418438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7396d-081b-4177-991d-1916756b2948 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.418712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7396d-081b-4177-991d-1916756b2948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.418980 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7396d-081b-4177-991d-1916756b2948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.427661 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7396d-081b-4177-991d-1916756b2948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:40.429100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7396d-081b-4177-991d-1916756b2948 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:40.440150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1322/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:40.619075 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:40.620711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:40.633347 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0c389-d5d5-4831-beec-277ea3477e3a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:40.638016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:40.638912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.639193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.640030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] PATH_INFO: `/v3/projects/01d7d32119984a1aacc7109e74d9fe01` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.641182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d7d32119984a1aacc7109e74d9fe01)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:40.643886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:40.705384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d7d32119984a1aacc7109e74d9fe01', 'request_id': 'req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.711235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d7d32119984a1aacc7109e74d9fe01', 'request_id': 'req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.719931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d7d32119984a1aacc7109e74d9fe01', 'request_id': 'req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.731495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] The token cache is being invalidate because project 01d7d32119984a1aacc7109e74d9fe01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:56:40.731790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.732424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9391b4f-4b0d-4ae3-802b-86fdf3d257c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:56:40.733908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1335/2657] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:56:40 2022] DELETE /identity/v3/projects/01d7d32119984a1aacc7109e74d9fe01 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:40.738709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.738991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.739261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.761431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:40.762635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:40.822382 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1323/2658] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:40.831376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.831692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.831967 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.855209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:40.857218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:40.866565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1336/2659] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:40.870806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:40.880961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251adf6f-f8bb-427a-9ae5-b1e9325a6f06 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:40.885619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:40.886208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.886488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.886757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.887560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:56:40.887860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:40.889431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:40.901514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1324/2660] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:56:40.905922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:40.916091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95279e-4fdd-4def-b63b-c865a48a5838 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:40.920806 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:40.921405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:40.921685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:40.921960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:40.922600 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:40.924671 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:40.956398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1337/2661] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:56:40 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:56:40.958657 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:40.961064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:40.961223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:41.021163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cfe63-07cb-4d91-9bc4-a47af9ad632c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:41.070997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:41.090588 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:41.091328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.091653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.091918 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.093887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:41.150878 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be4b72-529f-4348-94d2-834923280381 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:41.155522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:41.156465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.156465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.156789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.157422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:41.158416 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:41.159320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:41.159894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:41.169648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1325/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 00:56:41.180953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1338/2663] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:56:40 2022] POST /identity/v3/users => generated 296 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:56:41.185143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:41.195967 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b15ec-2050-4284-8491-3fc52641f67b service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:41.200605 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:41.201194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.201470 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.201745 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.202423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:41.204037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:41.214521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1326/2664] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:56:41.218466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:41.228867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3956e1-8eb4-46b6-aea9-7b9338ed91f3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:41.235294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897e257-5b85-4a69-8466-6ef7542a0301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:41.235953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897e257-5b85-4a69-8466-6ef7542a0301 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.236227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897e257-5b85-4a69-8466-6ef7542a0301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.236558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897e257-5b85-4a69-8466-6ef7542a0301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.237152 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897e257-5b85-4a69-8466-6ef7542a0301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:41.238906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897e257-5b85-4a69-8466-6ef7542a0301 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:41.240701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:41.251752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1339/2665] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:56:41.256930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:41.266708 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2712-4c6e-4fd3-925d-21a5e10e176b admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:41.271537 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:41.272787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.273058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.273332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] PATH_INFO: `/v3/projects/bc9e9218a2844f2ab3693f64a7538ad0/users/da3439df077749fa907c6619d17adbcb/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.289670 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9e9218a2844f2ab3693f64a7538ad0, user_id=da3439df077749fa907c6619d17adbcb, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:41.292409 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:41.306955 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1327/2666] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:56:41 2022] PUT /identity/v3/projects/bc9e9218a2844f2ab3693f64a7538ad0/users/da3439df077749fa907c6619d17adbcb/roles/87f1831257824560813f954348534540 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:56:41.373307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.373828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.373828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.404789 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] MFA Rules not processed for user `da3439df077749fa907c6619d17adbcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:56:41.406130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:41.471380 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1340/2667] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:56:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:56:41.482977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:56:41.484707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:56:41.542064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e576-45c4-44ce-9e80-b5be4cf69abd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:56:41.545936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5859dd-5036-44c2-af63-8d7784aa2902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:56:41.547044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5859dd-5036-44c2-af63-8d7784aa2902 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:56:41.547321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5859dd-5036-44c2-af63-8d7784aa2902 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:56:41.547630 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5859dd-5036-44c2-af63-8d7784aa2902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:56:41.557669 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5859dd-5036-44c2-af63-8d7784aa2902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:56:41.559083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5859dd-5036-44c2-af63-8d7784aa2902 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:56:41.567909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1328/2668] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:56:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:56:51.468326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=34 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:56:51.563852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:05.050032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:05.110595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ff2f-53db-4bfa-9dc1-13f9664bc33b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:05.115101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:05.115757 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:05.116033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:05.116304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] PATH_INFO: `/v3/users/da3439df077749fa907c6619d17adbcb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:05.117408 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3439df077749fa907c6619d17adbcb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:05.120047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:05.163860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3439df077749fa907c6619d17adbcb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.171346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3439df077749fa907c6619d17adbcb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.171976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3439df077749fa907c6619d17adbcb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.172566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3439df077749fa907c6619d17adbcb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.179296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3439df077749fa907c6619d17adbcb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.196644 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1341/2669] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:57:05 2022] DELETE /identity/v3/users/da3439df077749fa907c6619d17adbcb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:05.201092 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:05.201555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:05.201654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:05.220370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:05.221551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:05.278390 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1329/2670] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 00:57:05.288346 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:05.289957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:05.348082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917e338-d6ac-4ca0-a6cd-77b29399b06f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:05.351776 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc6808-6512-4882-beed-74c24e92d60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:05.352410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6808-6512-4882-beed-74c24e92d60e service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:05.352700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6808-6512-4882-beed-74c24e92d60e service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:05.353040 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6808-6512-4882-beed-74c24e92d60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:05.362245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc6808-6512-4882-beed-74c24e92d60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:05.363717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc6808-6512-4882-beed-74c24e92d60e service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:05.373641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1342/2671] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:05.651339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:05.652895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:05.666884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23051a9-4214-409f-a18f-19813d449d51 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:05.678000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:05.678639 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:05.678912 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:05.679322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] PATH_INFO: `/v3/projects/bc9e9218a2844f2ab3693f64a7538ad0` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:05.680569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9e9218a2844f2ab3693f64a7538ad0)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:05.683019 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:05.734604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9e9218a2844f2ab3693f64a7538ad0', 'request_id': 'req-8b7205aa-82c6-449a-a654-36f717eee8e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.741079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9e9218a2844f2ab3693f64a7538ad0', 'request_id': 'req-8b7205aa-82c6-449a-a654-36f717eee8e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.749939 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9e9218a2844f2ab3693f64a7538ad0', 'request_id': 'req-8b7205aa-82c6-449a-a654-36f717eee8e4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.761478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] The token cache is being invalidate because project bc9e9218a2844f2ab3693f64a7538ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:57:05.761767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.762389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:05.763959 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1330/2672] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:57:05 2022] DELETE /identity/v3/projects/bc9e9218a2844f2ab3693f64a7538ad0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:15.368850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:15.758592 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:31.218958 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:31.219448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:31.219587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:31.239373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] MFA Rules not processed for user `778fc9ee164e46a9b874dd0e20d38392`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:31.240603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:31.294895 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1343/2673] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:31.609800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:31.611264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:31.674876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7205aa-82c6-449a-a654-36f717eee8e4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:31.678787 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:31.679517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:31.679821 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:31.680151 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:31.682104 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:31.693240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:31.694926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:31.703471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1331/2674] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:38.999066 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:39.000522 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:39.056985 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968f615-ffea-4b29-8fd3-db1039409f8a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:39.060629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:39.061234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:39.061511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:39.061781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:39.063575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:39.144926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:39.146916 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:39.166513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1344/2675] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:41.699769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=26 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:47.277184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:47.285623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdbc9c-0e8f-446e-8015-975cd6878884 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:47.289210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:47.289792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:47.290063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:47.290325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:47.292147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:47.353037 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:47.354480 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:47.364458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1332/2676] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:49.163772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:50.533724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:50.535411 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:50.555203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be31f-41aa-4fa2-908f-0290d1f5cc06 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:50.560128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:50.560960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:50.561184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:50.561460 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] PATH_INFO: `/v3/users/36e8e45674624c219b59abb39d8dcf83` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:50.562700 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e8e45674624c219b59abb39d8dcf83)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:50.564723 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:50.623091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e8e45674624c219b59abb39d8dcf83'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.632543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e8e45674624c219b59abb39d8dcf83'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.633344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e8e45674624c219b59abb39d8dcf83'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.634799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e8e45674624c219b59abb39d8dcf83'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.649956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55305b8e-cb52-417b-b781-5c33581f6e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e8e45674624c219b59abb39d8dcf83'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.669679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1345/2677] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:57:50 2022] DELETE /identity/v3/users/36e8e45674624c219b59abb39d8dcf83 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:50.839443 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:50.850456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6912df6-59af-4b60-8e0b-7b3fa3696d47 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:50.910425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:50.911187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:50.911471 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:50.911771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] PATH_INFO: `/v3/projects/72d7dfe402c045edbca04c0e7eee05f9` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:50.912898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d7dfe402c045edbca04c0e7eee05f9)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:50.914797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:50.960059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d7dfe402c045edbca04c0e7eee05f9', 'request_id': 'req-6c5f0cb2-647a-45d6-af58-ca0205250b91'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.965583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d7dfe402c045edbca04c0e7eee05f9', 'request_id': 'req-6c5f0cb2-647a-45d6-af58-ca0205250b91'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.974089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d7dfe402c045edbca04c0e7eee05f9', 'request_id': 'req-6c5f0cb2-647a-45d6-af58-ca0205250b91'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.985194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] The token cache is being invalidate because project 72d7dfe402c045edbca04c0e7eee05f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:57:50.985478 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.986098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6c5f0cb2-647a-45d6-af58-ca0205250b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:50.988660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1333/2678] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:57:50 2022] DELETE /identity/v3/projects/72d7dfe402c045edbca04c0e7eee05f9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:50.993486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:50.993766 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:50.994036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.013659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:51.014834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:51.072049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1346/2679] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:51.079693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.079977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.080249 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.099855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:51.101051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:51.108883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1334/2680] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:51.112960 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.122462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cfe4f-b9b3-43a3-b5d1-d06f40794ce5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.126946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.127558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.127840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.128108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.128746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:57:51.129294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.130866 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.142117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1347/2681] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:57:51.146805 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.155962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ce2d0-bc7a-434e-8977-b670cf028794 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.160648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.161227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.161502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.161769 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.162401 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.164288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.179844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:51.193276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1335/2682] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:51.197364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.207817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9527413-6ca2-45ea-84b5-98732b0d9df3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.262627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.263221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.263525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.263802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.264429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.266362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.274622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:51.289544 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1348/2683] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:57:51.293651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.303758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2799452-bcd1-4bfd-baf4-55248550c07a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.308321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.308926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.309201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.309469 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.310110 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.311718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.321568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1336/2684] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:51.325367 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.335904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b01ffd-f8b8-49be-b87e-21c595d0d29c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.340648 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ea363-f79a-476c-bf42-ebe1ae6df0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.341247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ea363-f79a-476c-bf42-ebe1ae6df0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.341526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ea363-f79a-476c-bf42-ebe1ae6df0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.341799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ea363-f79a-476c-bf42-ebe1ae6df0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.342461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ea363-f79a-476c-bf42-ebe1ae6df0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.344068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ea363-f79a-476c-bf42-ebe1ae6df0f7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.355964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1349/2685] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:51.360033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.369929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10b4c9-9dd1-41c3-bcac-c105ccb12d09 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.374616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.375251 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.375552 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.375825 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] PATH_INFO: `/v3/projects/c653382fde154db092256789e04ab82a/users/b9a3468e49ae4807842e34602ae906ec/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.393075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c653382fde154db092256789e04ab82a, user_id=b9a3468e49ae4807842e34602ae906ec, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.396312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.409225 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1337/2686] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:51 2022] PUT /identity/v3/projects/c653382fde154db092256789e04ab82a/users/b9a3468e49ae4807842e34602ae906ec/roles/87f1831257824560813f954348534540 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:51.420412 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.420699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.420973 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.448575 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] MFA Rules not processed for user `b9a3468e49ae4807842e34602ae906ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:51.449779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:51.513077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1350/2687] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:51.517691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.577869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d088759-1617-41bd-9012-b608272aae8a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.583006 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.583620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.583894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.584236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.584867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.586738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.613529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1338/2688] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:51.617616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.628651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5067631-158d-43f1-b6f7-a416c26f61ae None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.684654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.685255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.685529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.685797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.686414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.688362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.709685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1351/2689] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:51.713758 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.723919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff3814-1388-4094-9703-8627951a141b admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.728468 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.729059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.729336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.729601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.730236 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.732112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.742718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1339/2690] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:57:51.746886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.757856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4fdb-00dd-4881-aca8-7766e6ad8a41 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.762672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.763318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.763629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.763899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] PATH_INFO: `/v3/projects/f71a6c70076f43d5bd33ff99153cf600/users/aa0a506c22de4704a646645a07825ce0/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.779515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71a6c70076f43d5bd33ff99153cf600, user_id=aa0a506c22de4704a646645a07825ce0, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.782187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.796228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1352/2691] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:51 2022] PUT /identity/v3/projects/f71a6c70076f43d5bd33ff99153cf600/users/aa0a506c22de4704a646645a07825ce0/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:51.800232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.810128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95cc01-20b3-4ef8-9554-ca6e77a6f38d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.866896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac383a30-cc7e-4991-b29d-d5f213a1013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.867642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac383a30-cc7e-4991-b29d-d5f213a1013f admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.867775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac383a30-cc7e-4991-b29d-d5f213a1013f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.868054 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac383a30-cc7e-4991-b29d-d5f213a1013f admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.869201 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac383a30-cc7e-4991-b29d-d5f213a1013f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.870899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac383a30-cc7e-4991-b29d-d5f213a1013f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.884007 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1340/2692] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:51.888154 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:51.898328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee669da6-7c66-4da5-8154-350742797a60 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:51.902946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:51.903620 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.903897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.904196 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] PATH_INFO: `/v3/projects/f71a6c70076f43d5bd33ff99153cf600/users/aa0a506c22de4704a646645a07825ce0/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.906093 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71a6c70076f43d5bd33ff99153cf600, user_id=aa0a506c22de4704a646645a07825ce0, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:51.908818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:51.919026 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1353/2693] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:51 2022] PUT /identity/v3/projects/f71a6c70076f43d5bd33ff99153cf600/users/aa0a506c22de4704a646645a07825ce0/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:51.930833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:51.931198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:51.931690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:51.959943 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] MFA Rules not processed for user `aa0a506c22de4704a646645a07825ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:51.961152 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:51.980624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:52.032650 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1341/2694] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:52.046627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:52.048384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:52.072482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:52.110141 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e67b-15fd-4358-a498-9c37da674199 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:52.114399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:52.115000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:52.115279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:52.115574 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:52.124611 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:52.125979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:52.136208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1354/2695] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:52.239752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:52.249275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac1295-0cbd-4d51-ab04-2164e1870e9a None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:52.252843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:52.253936 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:52.254467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:52.254799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:52.313490 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:52.314913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:52.324532 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1342/2696] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:52.788910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:52.790320 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:52.857602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4cd18-191d-4e8b-8fe9-5df4e4240acf service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:52.862939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:52.863562 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:52.863839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:52.864105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:52.865830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:52.876809 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:52.878942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a85dee-06a3-4cda-a0f9-c8f1efb77b55 service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:52.893291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1355/2697] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:54.899742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:54.960033 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55b09-c5ef-4389-8003-7ea6007099eb service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:54.964441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:54.965056 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:54.965331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:54.965598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] PATH_INFO: `/v3/users/b9a3468e49ae4807842e34602ae906ec` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:54.966697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a3468e49ae4807842e34602ae906ec)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:54.968615 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:55.015246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a3468e49ae4807842e34602ae906ec'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.022736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a3468e49ae4807842e34602ae906ec'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.023370 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a3468e49ae4807842e34602ae906ec'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.025389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:55.031728 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a3468e49ae4807842e34602ae906ec'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.032674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a3468e49ae4807842e34602ae906ec'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.049448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1343/2698] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:57:54 2022] DELETE /identity/v3/users/b9a3468e49ae4807842e34602ae906ec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:55.054430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:55.054912 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:55.055032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:55.074314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:55.075558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:55.132560 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1356/2699] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:55.142324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:55.144312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:55.200978 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d99eb-bd52-44e7-ab02-cb37938d0e49 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:55.204602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:55.205190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:55.205465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:55.205732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:55.214550 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:55.215957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:55.225619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1344/2700] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:55.492784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:55.494228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:55.506877 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d9899-5513-4c30-8ff0-9572de90683d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:55.511513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:55.512133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:55.512409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:55.512679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] PATH_INFO: `/v3/projects/c653382fde154db092256789e04ab82a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:55.513824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c653382fde154db092256789e04ab82a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:55.515756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:55.562086 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:55.562937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c653382fde154db092256789e04ab82a', 'request_id': 'req-0027d4c7-087d-4a58-b23e-e8831406b4e4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.569446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c653382fde154db092256789e04ab82a', 'request_id': 'req-0027d4c7-087d-4a58-b23e-e8831406b4e4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.576781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c653382fde154db092256789e04ab82a', 'request_id': 'req-0027d4c7-087d-4a58-b23e-e8831406b4e4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.587692 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] The token cache is being invalidate because project c653382fde154db092256789e04ab82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:57:55.587938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.588553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.590021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1357/2701] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:57:55 2022] DELETE /identity/v3/projects/c653382fde154db092256789e04ab82a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:55.594140 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:55.604212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5241-03ab-48af-bf3b-3691abdd8d8c service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:55.658376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:55.658994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:55.659269 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:55.660190 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] PATH_INFO: `/v3/users/aa0a506c22de4704a646645a07825ce0` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:55.661293 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0a506c22de4704a646645a07825ce0)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:55.663174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:55.708594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0a506c22de4704a646645a07825ce0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.715851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0a506c22de4704a646645a07825ce0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.716546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0a506c22de4704a646645a07825ce0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.724253 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0a506c22de4704a646645a07825ce0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.724872 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a9f69f3a-1156-4c61-98e0-8c0a5a7992e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0a506c22de4704a646645a07825ce0'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:55.735653 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:55.744315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1345/2702] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:57:55 2022] DELETE /identity/v3/users/aa0a506c22de4704a646645a07825ce0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:56.011083 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.012656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:56.027019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027d4c7-087d-4a58-b23e-e8831406b4e4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.089462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.090082 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.090363 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.090636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] PATH_INFO: `/v3/projects/f71a6c70076f43d5bd33ff99153cf600` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.091799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71a6c70076f43d5bd33ff99153cf600)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.093708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.139737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71a6c70076f43d5bd33ff99153cf600', 'request_id': 'req-db74f33a-f250-447a-b825-29a3a85c4655'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:56.145134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71a6c70076f43d5bd33ff99153cf600', 'request_id': 'req-db74f33a-f250-447a-b825-29a3a85c4655'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:56.152057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71a6c70076f43d5bd33ff99153cf600', 'request_id': 'req-db74f33a-f250-447a-b825-29a3a85c4655'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:56.163296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] The token cache is being invalidate because project f71a6c70076f43d5bd33ff99153cf600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:57:56.163608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:56.164231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db74f33a-f250-447a-b825-29a3a85c4655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:57:56.165660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1358/2703] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:57:56 2022] DELETE /identity/v3/projects/f71a6c70076f43d5bd33ff99153cf600 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:56.170603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.171120 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.171223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.190977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:56.192224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:56.248677 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1346/2704] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:56.257102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560f872-791b-4803-bc23-820a90bd84da None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.257325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560f872-791b-4803-bc23-820a90bd84da None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.257596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560f872-791b-4803-bc23-820a90bd84da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.277701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7560f872-791b-4803-bc23-820a90bd84da None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:56.278890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7560f872-791b-4803-bc23-820a90bd84da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:56.286754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1359/2705] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:56.291014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.300660 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e5f38-f4dd-421d-a295-284eabc32a99 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.305276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.305862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.306141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.306414 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.307060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:57:56.307358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.308964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.320736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1347/2706] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:57:56.324847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560f872-791b-4803-bc23-820a90bd84da None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.334311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560f872-791b-4803-bc23-820a90bd84da None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.338959 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.339559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.339832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.340096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.340717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.342576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.369646 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1360/2707] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:56.373623 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.383850 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53d2ee-78f0-4564-9bed-c1060fe3fd3a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.438755 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.439345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.439659 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.439929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.440560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.442479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.463224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1348/2708] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:56.467246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.477995 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a319b0-7512-4921-96e8-8d29c83a192f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.482629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.483214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.483500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.483888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.484558 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.486200 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.496063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1361/2709] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:56.499887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.509961 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7709d-9d4a-47ed-a8d8-929c395f9574 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.514439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314e820-a0e6-45e4-9520-449afc5e0f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.515023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314e820-a0e6-45e4-9520-449afc5e0f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.515302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314e820-a0e6-45e4-9520-449afc5e0f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.515679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314e820-a0e6-45e4-9520-449afc5e0f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.516319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314e820-a0e6-45e4-9520-449afc5e0f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.517895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314e820-a0e6-45e4-9520-449afc5e0f89 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.529024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1349/2710] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:56.532937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.542488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4095c2c-3c18-4cbe-bab2-443e067b1ce4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.546976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.547656 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.547932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.548201 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] PATH_INFO: `/v3/projects/04ba414067904f2eb6cf067ea547fb87/users/9fcc57bebe934fcab36a036e15f94e99/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.562635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ba414067904f2eb6cf067ea547fb87, user_id=9fcc57bebe934fcab36a036e15f94e99, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.565292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.578565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1362/2711] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:56 2022] PUT /identity/v3/projects/04ba414067904f2eb6cf067ea547fb87/users/9fcc57bebe934fcab36a036e15f94e99/roles/87f1831257824560813f954348534540 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:56.588637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.588919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.589189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.616452 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] MFA Rules not processed for user `9fcc57bebe934fcab36a036e15f94e99`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:56.618102 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:56.666209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:56.684750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1350/2712] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:56.689244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.693747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:56.753965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46184705-3a32-4762-88c3-6c1583234e75 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.758762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.759351 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.759660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.759929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.760604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.762430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.788136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1363/2713] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:56.792395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.802711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deaa06-c6a1-403d-8d71-ce9805962542 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.858325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.858923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.859198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.859465 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.860186 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.863514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.885568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1351/2714] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:57:56 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:56.889482 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.900147 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71569597-30a0-4b51-80fa-5b2a162d039a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.904803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.905386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.905659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.905923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.906661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.908281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.918454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1364/2715] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:56.922282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.932343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4165-09dd-4d31-8974-1f780a84b89f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.936877 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a0b75-b2e8-4674-87cb-05c0882ccac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.937454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a0b75-b2e8-4674-87cb-05c0882ccac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.937726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a0b75-b2e8-4674-87cb-05c0882ccac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.937990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a0b75-b2e8-4674-87cb-05c0882ccac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.938614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a0b75-b2e8-4674-87cb-05c0882ccac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.940222 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a0b75-b2e8-4674-87cb-05c0882ccac7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:56.950385 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1352/2716] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:56.954541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:56.965056 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e8b3-42cd-4121-8719-fc40bcaafd9b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:56.969559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:56.970190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:56.970461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:56.970729 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] PATH_INFO: `/v3/projects/0eb3992f376a439487d83ba5ddc50fcd/users/ee8808620dd64160bd0706ae5e5aa2a8/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:56.984903 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb3992f376a439487d83ba5ddc50fcd, user_id=ee8808620dd64160bd0706ae5e5aa2a8, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:56.987664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.000223 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1365/2717] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:56 2022] PUT /identity/v3/projects/0eb3992f376a439487d83ba5ddc50fcd/users/ee8808620dd64160bd0706ae5e5aa2a8/roles/87f1831257824560813f954348534540 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:57.012329 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.012553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.012822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.040702 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] MFA Rules not processed for user `ee8808620dd64160bd0706ae5e5aa2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:57.041910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:57.103724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1353/2718] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:57.108287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.168065 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ba387-3439-4319-af72-5f46ff0a9603 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.172596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.173181 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.173456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.173720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.174328 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.176742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.202865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1366/2719] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:57:57 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:57.207124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.218177 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84642f-cdb4-41dd-812a-6c60c6b81787 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.273698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.274211 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.274483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.274748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.275365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.277291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.285008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:57.298876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1354/2720] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:57:57 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:57:57.303455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.313554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90534334-96d0-4aed-9f1b-d2216d5f62f5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.318682 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.319264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.319572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.319843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.320488 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.322096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.333261 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1367/2721] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:57:57.337304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.347311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8ed97-10e4-41d7-8f7b-641d2f23c1b5 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.351938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.352578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.352854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.353124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] PATH_INFO: `/v3/projects/ecb8581d19e645419cca973afb452c9d/users/5b93a467da3740aa86498642b610dca4/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.367910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb8581d19e645419cca973afb452c9d, user_id=5b93a467da3740aa86498642b610dca4, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.370529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.383270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1355/2722] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:57 2022] PUT /identity/v3/projects/ecb8581d19e645419cca973afb452c9d/users/5b93a467da3740aa86498642b610dca4/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:57.387071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.397228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57520037-1071-4e6f-82e7-e8db987937cd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.419849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:57:57.452779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de07d1-ad91-4a96-9973-fffc052fe322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.453364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de07d1-ad91-4a96-9973-fffc052fe322 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.453637 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de07d1-ad91-4a96-9973-fffc052fe322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.453904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de07d1-ad91-4a96-9973-fffc052fe322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.454533 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de07d1-ad91-4a96-9973-fffc052fe322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.456675 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de07d1-ad91-4a96-9973-fffc052fe322 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.466838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1368/2723] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 00:57:57.471268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.482199 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f566e-fb3f-419f-9d5b-e226b7c705da admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.486779 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.487435 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.487743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.488016 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] PATH_INFO: `/v3/projects/ecb8581d19e645419cca973afb452c9d/users/5b93a467da3740aa86498642b610dca4/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.489916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb8581d19e645419cca973afb452c9d, user_id=5b93a467da3740aa86498642b610dca4, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.492638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.504522 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1356/2724] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:57:57 2022] PUT /identity/v3/projects/ecb8581d19e645419cca973afb452c9d/users/5b93a467da3740aa86498642b610dca4/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:57:57.514681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.514956 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.515332 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.542932 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] MFA Rules not processed for user `5b93a467da3740aa86498642b610dca4`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:57:57.544076 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:57.609802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1369/2725] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:57:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:57:57.620875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.622572 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:57.680339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1d92-9ef4-4c68-9ff5-8f1272209530 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.683987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.684587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.684866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.685133 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.693957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.695384 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.704793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1357/2726] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:57:57.746867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:57:57.748329 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:57:57.760568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f889e4-2516-4227-9f7b-dc969a7c2f4b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:57:57.764262 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992ca31-7248-461e-b5b9-c4b6c237301f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:57:57.764908 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992ca31-7248-461e-b5b9-c4b6c237301f service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:57:57.765184 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992ca31-7248-461e-b5b9-c4b6c237301f service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:57:57.765454 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992ca31-7248-461e-b5b9-c4b6c237301f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:57:57.821156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992ca31-7248-461e-b5b9-c4b6c237301f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:57:57.822738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992ca31-7248-461e-b5b9-c4b6c237301f service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:57:57.831543 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1370/2727] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:57:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:00.685712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:00.746001 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27506ab-2a1a-44fd-b21f-816250c692b1 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:00.750741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:00.751360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:00.751674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:00.751975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] PATH_INFO: `/v3/users/9fcc57bebe934fcab36a036e15f94e99` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:00.753579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fcc57bebe934fcab36a036e15f94e99)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:00.755752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:00.801432 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fcc57bebe934fcab36a036e15f94e99'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:00.807827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fcc57bebe934fcab36a036e15f94e99'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:00.808489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fcc57bebe934fcab36a036e15f94e99'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:00.810411 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:00.817483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fcc57bebe934fcab36a036e15f94e99'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:00.818264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fcc57bebe934fcab36a036e15f94e99'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:00.836205 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1358/2728] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:00 2022] DELETE /identity/v3/users/9fcc57bebe934fcab36a036e15f94e99 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:00.840663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:00.841132 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:00.841234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:00.859621 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:00.860743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:00.901451 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:00.918513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1371/2729] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:00.928510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:00.929938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:00.985693 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a05d87-1a18-4d66-a459-5a3edb3709f9 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:00.989854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:00.990447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:00.990726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:00.990997 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:00.999916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:01.001328 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:01.011529 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1359/2730] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:01.280879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:01.282450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:01.294608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2d23f-1e00-4ae8-95d3-69da2dbd7619 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:01.299139 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:01.299953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:01.300229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:01.300489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] PATH_INFO: `/v3/projects/04ba414067904f2eb6cf067ea547fb87` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:01.301615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ba414067904f2eb6cf067ea547fb87)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:01.303520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:01.349790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ba414067904f2eb6cf067ea547fb87', 'request_id': 'req-2d45827e-6068-4693-82c0-6018171b4c6d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.355297 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ba414067904f2eb6cf067ea547fb87', 'request_id': 'req-2d45827e-6068-4693-82c0-6018171b4c6d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.362210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ba414067904f2eb6cf067ea547fb87', 'request_id': 'req-2d45827e-6068-4693-82c0-6018171b4c6d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.372966 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] The token cache is being invalidate because project 04ba414067904f2eb6cf067ea547fb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:01.373251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.373870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.375331 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1372/2731] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:01 2022] DELETE /identity/v3/projects/04ba414067904f2eb6cf067ea547fb87 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:01.380053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:01.390764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed92c68-13bf-49e2-a8a7-1755db871271 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:01.446265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:01.446817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:01.447094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:01.447365 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] PATH_INFO: `/v3/users/ee8808620dd64160bd0706ae5e5aa2a8` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:01.448503 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee8808620dd64160bd0706ae5e5aa2a8)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:01.450406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:01.495977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee8808620dd64160bd0706ae5e5aa2a8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.503923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee8808620dd64160bd0706ae5e5aa2a8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.504778 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee8808620dd64160bd0706ae5e5aa2a8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.511571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee8808620dd64160bd0706ae5e5aa2a8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.512194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee8808620dd64160bd0706ae5e5aa2a8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.523440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:01.531332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1360/2732] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:01 2022] DELETE /identity/v3/users/ee8808620dd64160bd0706ae5e5aa2a8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:01.800377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:01.801933 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:01.815500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45827e-6068-4693-82c0-6018171b4c6d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:01.871888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:01.872547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:01.872765 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:01.873030 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] PATH_INFO: `/v3/projects/0eb3992f376a439487d83ba5ddc50fcd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:01.874124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb3992f376a439487d83ba5ddc50fcd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:01.876058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:01.926214 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb3992f376a439487d83ba5ddc50fcd', 'request_id': 'req-5daea729-c628-4f48-a8ad-534ea431dfdb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.931811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb3992f376a439487d83ba5ddc50fcd', 'request_id': 'req-5daea729-c628-4f48-a8ad-534ea431dfdb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.939717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb3992f376a439487d83ba5ddc50fcd', 'request_id': 'req-5daea729-c628-4f48-a8ad-534ea431dfdb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.947269 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:01.951354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] The token cache is being invalidate because project 0eb3992f376a439487d83ba5ddc50fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:01.951815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.952871 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:01.954601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1373/2733] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:01 2022] DELETE /identity/v3/projects/0eb3992f376a439487d83ba5ddc50fcd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:01.959298 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:01.960725 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:01.974023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22724-b746-41f9-ba49-3f1dca400e0c admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.028611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.029161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.029438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.029710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] PATH_INFO: `/v3/users/5b93a467da3740aa86498642b610dca4` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.030815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b93a467da3740aa86498642b610dca4)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.032743 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.077070 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b93a467da3740aa86498642b610dca4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.084844 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b93a467da3740aa86498642b610dca4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.085574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b93a467da3740aa86498642b610dca4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.092618 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b93a467da3740aa86498642b610dca4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.093261 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-cfe5449c-9e83-4b0c-89e9-e6ab3fa7cba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b93a467da3740aa86498642b610dca4'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.109932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1361/2734] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:01 2022] DELETE /identity/v3/users/5b93a467da3740aa86498642b610dca4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:02.386439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.387936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:02.401281 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea729-c628-4f48-a8ad-534ea431dfdb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.457946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.458572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.458850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.459124 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] PATH_INFO: `/v3/projects/ecb8581d19e645419cca973afb452c9d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.460298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb8581d19e645419cca973afb452c9d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.462208 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.507750 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb8581d19e645419cca973afb452c9d', 'request_id': 'req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.513832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb8581d19e645419cca973afb452c9d', 'request_id': 'req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.523850 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb8581d19e645419cca973afb452c9d', 'request_id': 'req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.538790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] The token cache is being invalidate because project ecb8581d19e645419cca973afb452c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:02.539078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.539752 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9fd665ab-8bdf-42c9-93bf-f50cac68ced9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:02.541237 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1374/2735] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:02 2022] DELETE /identity/v3/projects/ecb8581d19e645419cca973afb452c9d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:02.547116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.547398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.547697 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.566995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:02.568148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:02.625695 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1362/2736] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:02.634245 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.634526 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.634800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.655664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:02.656864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:02.665431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1375/2737] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:02.669752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.679585 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bb1df-052f-48d8-a93d-06d60d89f03d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.684059 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.684647 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.684921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.685188 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.685827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:58:02.686126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.687722 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.698924 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1363/2738] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:58:02.703233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.712745 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15aef9-6964-4c43-b508-f16e844ae20d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.717314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.717893 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.718164 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.718429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.719050 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.720945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.746714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1376/2739] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:58:02 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:58:02.751014 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.757052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:02.763058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1625b-03dc-4d2c-be15-e83cf8a52ea3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.818057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.818581 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.818849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.819124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.819784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.821672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.842710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1364/2740] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:58:02 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:58:02.847188 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.859213 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d97ff-5f65-41b3-9d01-4299c614ed06 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.864469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.865064 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.865336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.865605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.866309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.867990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.879094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1377/2741] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:02.883356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.893148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c7cc3-37e0-41ed-86e3-11de7d5eb92d admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.898398 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7aefd-642d-4fea-b8c9-fcefc0ac1f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.898982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7aefd-642d-4fea-b8c9-fcefc0ac1f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.899258 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7aefd-642d-4fea-b8c9-fcefc0ac1f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.899561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7aefd-642d-4fea-b8c9-fcefc0ac1f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.900196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7aefd-642d-4fea-b8c9-fcefc0ac1f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.901763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7aefd-642d-4fea-b8c9-fcefc0ac1f3b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.912053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1365/2742] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:02.915928 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:02.925613 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7e1d7-d88d-41a4-8f78-865b994ce74b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:02.930311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:02.930941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.931230 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.931501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] PATH_INFO: `/v3/projects/983bb4fc3ece437cbf8dffe119eab3fe/users/3817ca05ee1a462d8a0ab50fa3dfdf0f/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:02.945859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] RBAC: Authorizing `identity:create_grant(project_id=983bb4fc3ece437cbf8dffe119eab3fe, user_id=3817ca05ee1a462d8a0ab50fa3dfdf0f, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:02.948520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:02.961052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1378/2743] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:58:02 2022] PUT /identity/v3/projects/983bb4fc3ece437cbf8dffe119eab3fe/users/3817ca05ee1a462d8a0ab50fa3dfdf0f/roles/87f1831257824560813f954348534540 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:02.972905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:02.973403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:02.973520 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.000712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] MFA Rules not processed for user `3817ca05ee1a462d8a0ab50fa3dfdf0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:03.001925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:03.063417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1366/2744] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:03.068391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.097740 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:03.131407 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830ffe1-ebfd-4268-8225-3f65adc8047d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.135942 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.136528 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.136801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.137096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.137704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.139672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.165352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1379/2745] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:58:03 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:58:03.169404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.180460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88d3e6-c331-4090-8d96-2632da781e5c None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.236117 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.236726 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.237009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.237766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.238399 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.240602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.257492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:03.263410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1367/2746] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:58:03 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:58:03.267379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.277516 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e099f-8b18-4312-bca7-887546f58e37 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.282122 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.282717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.282996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.283270 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.283945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.285572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.295827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1380/2747] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:03.300289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.310134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c531f-13c4-4a52-93ae-5230286d32b1 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.314717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.315354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.316193 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.316467 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] PATH_INFO: `/v3/projects/75210461805046b895f921b4c20d9c50/users/6e4f350e3437425cbf883118f6ddea2b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.330587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75210461805046b895f921b4c20d9c50, user_id=6e4f350e3437425cbf883118f6ddea2b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.333264 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.345280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1368/2748] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:58:03 2022] PUT /identity/v3/projects/75210461805046b895f921b4c20d9c50/users/6e4f350e3437425cbf883118f6ddea2b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:03.349227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.359271 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa0aeb-0f3a-4a13-9c3e-02748c91bb00 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.415189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e82992-ef90-42df-a60d-bf693b48f255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.415865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e82992-ef90-42df-a60d-bf693b48f255 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.416151 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e82992-ef90-42df-a60d-bf693b48f255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.416423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e82992-ef90-42df-a60d-bf693b48f255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.417068 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e82992-ef90-42df-a60d-bf693b48f255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.418693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e82992-ef90-42df-a60d-bf693b48f255 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.428203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1381/2749] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:03.432276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.442407 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd6e8-9182-4a89-a3ed-8f7fb90f17b4 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.446988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.447667 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.447944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.448213 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] PATH_INFO: `/v3/projects/75210461805046b895f921b4c20d9c50/users/6e4f350e3437425cbf883118f6ddea2b/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.450223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75210461805046b895f921b4c20d9c50, user_id=6e4f350e3437425cbf883118f6ddea2b, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.452963 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.463578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1369/2750] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:58:03 2022] PUT /identity/v3/projects/75210461805046b895f921b4c20d9c50/users/6e4f350e3437425cbf883118f6ddea2b/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:58:03.473798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.474078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.474349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.500900 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] MFA Rules not processed for user `6e4f350e3437425cbf883118f6ddea2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:03.502092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:03.567321 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1382/2751] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:03.578218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:03.579926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:03.637354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ba96-73f3-4e2f-9c01-c3874e5c3a14 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:03.640930 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:03.641528 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:03.641809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:03.642081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:03.697389 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:03.698813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:03.708166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1370/2752] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:07.913896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:07.915311 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:07.927080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a77d8f-d012-4691-a4e1-44a8a85415b0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:07.930567 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83fedc-550f-4a4a-8dd4-262c963bb2b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:07.931148 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83fedc-550f-4a4a-8dd4-262c963bb2b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:07.931423 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83fedc-550f-4a4a-8dd4-262c963bb2b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:07.931727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83fedc-550f-4a4a-8dd4-262c963bb2b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:07.940345 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83fedc-550f-4a4a-8dd4-262c963bb2b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:07.941756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83fedc-550f-4a4a-8dd4-262c963bb2b9 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:07.950939 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1383/2753] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:12.819771 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:12.882112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017ceb4-0f2c-445b-b6c1-92b3a71572c1 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:12.886690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:12.887311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:12.887619 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:12.887893 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] PATH_INFO: `/v3/users/3817ca05ee1a462d8a0ab50fa3dfdf0f` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:12.889013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3817ca05ee1a462d8a0ab50fa3dfdf0f)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:12.890960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:12.911441 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:12.937425 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3817ca05ee1a462d8a0ab50fa3dfdf0f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:12.944689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3817ca05ee1a462d8a0ab50fa3dfdf0f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:12.945268 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3817ca05ee1a462d8a0ab50fa3dfdf0f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:12.952088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3817ca05ee1a462d8a0ab50fa3dfdf0f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:12.952706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3817ca05ee1a462d8a0ab50fa3dfdf0f'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:12.969375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1371/2754] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:12 2022] DELETE /identity/v3/users/3817ca05ee1a462d8a0ab50fa3dfdf0f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:12.973902 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:12.974437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:12.974437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:12.993250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:12.994426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:12.998199 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:13.053302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1384/2755] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:13.063071 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:13.064507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:13.120306 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca44673-9340-4ca0-aceb-e6e2104b9569 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:13.123971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:13.124569 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:13.124848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:13.125119 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:13.134224 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:13.135665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:13.145614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1372/2756] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:13.404396 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:13.405814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:13.417918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31878a-8feb-4e3f-a75e-077bb3d6ea8e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:13.422550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:13.423182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:13.423461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:13.423762 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] PATH_INFO: `/v3/projects/983bb4fc3ece437cbf8dffe119eab3fe` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:13.424868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=983bb4fc3ece437cbf8dffe119eab3fe)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:13.426922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:13.473120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983bb4fc3ece437cbf8dffe119eab3fe', 'request_id': 'req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.478681 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983bb4fc3ece437cbf8dffe119eab3fe', 'request_id': 'req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.485839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983bb4fc3ece437cbf8dffe119eab3fe', 'request_id': 'req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.496643 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] The token cache is being invalidate because project 983bb4fc3ece437cbf8dffe119eab3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:13.496931 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.497554 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.499052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1385/2757] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:13 2022] DELETE /identity/v3/projects/983bb4fc3ece437cbf8dffe119eab3fe => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:13.503685 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:13.513611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fcf38-da52-4da3-8dd5-cabd7d34eb43 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:13.568267 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:13.568882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:13.569161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:13.569437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] PATH_INFO: `/v3/users/6e4f350e3437425cbf883118f6ddea2b` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:13.570548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4f350e3437425cbf883118f6ddea2b)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:13.572477 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:13.613128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:13.618822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4f350e3437425cbf883118f6ddea2b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.626297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4f350e3437425cbf883118f6ddea2b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.626899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4f350e3437425cbf883118f6ddea2b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.634712 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4f350e3437425cbf883118f6ddea2b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.635338 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-66dcbe4e-1192-454f-9e2a-d6bef726a33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4f350e3437425cbf883118f6ddea2b'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:13.652252 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1373/2758] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:13 2022] DELETE /identity/v3/users/6e4f350e3437425cbf883118f6ddea2b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:13.919357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:13.920831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:13.936976 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a6e13-e478-468f-b2fb-b14e4c6c08ce admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:13.995775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:13.996443 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:13.996689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:13.997005 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] PATH_INFO: `/v3/projects/75210461805046b895f921b4c20d9c50` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:13.998045 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=75210461805046b895f921b4c20d9c50)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:13.999980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.036780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:14.047569 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75210461805046b895f921b4c20d9c50', 'request_id': 'req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.053734 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75210461805046b895f921b4c20d9c50', 'request_id': 'req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.061601 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75210461805046b895f921b4c20d9c50', 'request_id': 'req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.072195 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] The token cache is being invalidate because project 75210461805046b895f921b4c20d9c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:14.072478 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.073103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3b6d1cfe-ec53-4aac-b7f4-42e461c1acfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.074559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1386/2759] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:13 2022] DELETE /identity/v3/projects/75210461805046b895f921b4c20d9c50 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:14.080400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.080905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.081047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.100942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:14.102154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:14.160068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1374/2760] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:14.224241 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.224519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.224816 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.245246 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:14.246462 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:14.254416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1387/2761] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:14.258809 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.268431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d418df6-5c36-44fa-87ff-635c74ca75ff None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.272900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.273482 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.273868 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.274142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.274781 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:58:14.275079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.276682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.288302 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1375/2762] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:58:14.292518 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.302105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d263cc4-a182-4d90-9331-3b06f9ab6999 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.306640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.307216 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.307501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.307794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.308418 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.310303 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.336479 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1388/2763] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:58:14 2022] POST /identity/v3/projects => generated 359 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:58:14.340535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.350951 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1414-d016-4785-a75b-d0050cb3ddf3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.405063 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.405655 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.405933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.406200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.406818 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.408736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.430262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1376/2764] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:58:14 2022] POST /identity/v3/users => generated 280 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 00:58:14.434306 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.444398 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb73a2-0684-42aa-8d3e-0e2f0c198852 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.449501 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.450089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.450366 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.450636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.451274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.452898 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.463191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1389/2765] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:14.467068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.477324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a05f-73bc-4723-ae9a-6fa5f5f259d0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.481876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a50bff-9cba-4ab2-9360-3f7f8ed5e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.482457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a50bff-9cba-4ab2-9360-3f7f8ed5e177 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.482735 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a50bff-9cba-4ab2-9360-3f7f8ed5e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.483010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a50bff-9cba-4ab2-9360-3f7f8ed5e177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.483674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a50bff-9cba-4ab2-9360-3f7f8ed5e177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.485238 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a50bff-9cba-4ab2-9360-3f7f8ed5e177 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.495498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1377/2766] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:14.499350 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.509426 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e120510-fb70-4ed6-b913-7f8b53345af4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.513924 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.514550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.514825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.515092 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] PATH_INFO: `/v3/projects/82003c14585048029e15b38fc706236d/users/2dba1d8463514362bd0c9eca767f640a/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.529659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82003c14585048029e15b38fc706236d, user_id=2dba1d8463514362bd0c9eca767f640a, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.532314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.546178 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1390/2767] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:58:14 2022] PUT /identity/v3/projects/82003c14585048029e15b38fc706236d/users/2dba1d8463514362bd0c9eca767f640a/roles/87f1831257824560813f954348534540 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:14.556878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85594354-da73-4747-abca-da2bb645fed2 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.557164 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85594354-da73-4747-abca-da2bb645fed2 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.557464 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85594354-da73-4747-abca-da2bb645fed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.584767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-85594354-da73-4747-abca-da2bb645fed2 None None] MFA Rules not processed for user `2dba1d8463514362bd0c9eca767f640a`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:14.585975 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-85594354-da73-4747-abca-da2bb645fed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:14.598266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:14.653206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1378/2768] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:14.688964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.691294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:14.746677 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2b0db-b5fd-4640-bab5-6846797e1b32 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.750225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074220bb-7141-4c1f-a7a3-82b8d5d6ab16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.750815 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074220bb-7141-4c1f-a7a3-82b8d5d6ab16 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.751088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074220bb-7141-4c1f-a7a3-82b8d5d6ab16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.751355 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074220bb-7141-4c1f-a7a3-82b8d5d6ab16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.759801 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074220bb-7141-4c1f-a7a3-82b8d5d6ab16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.761167 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074220bb-7141-4c1f-a7a3-82b8d5d6ab16 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.769920 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1391/2769] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:14.785539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85594354-da73-4747-abca-da2bb645fed2 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:14.845404 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85594354-da73-4747-abca-da2bb645fed2 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:14.850424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:14.851034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.851315 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.851613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] PATH_INFO: `/v3/users/2dba1d8463514362bd0c9eca767f640a` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.852724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dba1d8463514362bd0c9eca767f640a)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:14.854609 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:14.897822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dba1d8463514362bd0c9eca767f640a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.904270 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dba1d8463514362bd0c9eca767f640a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.904942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dba1d8463514362bd0c9eca767f640a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.912651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dba1d8463514362bd0c9eca767f640a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.913287 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dba1d8463514362bd0c9eca767f640a'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:14.930770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1379/2770] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:14 2022] DELETE /identity/v3/users/2dba1d8463514362bd0c9eca767f640a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:14.935204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:14.935499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:14.935791 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:14.954510 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:14.955796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:15.013011 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1392/2771] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:15.022558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.023994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:15.059819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:15.082548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a5c5d-7ef9-4b0b-afd1-06d7ec69bbf7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.086208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ff91d-ecc1-4538-bb04-6710a9cd2078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.086816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ff91d-ecc1-4538-bb04-6710a9cd2078 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.087095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ff91d-ecc1-4538-bb04-6710a9cd2078 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.087369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ff91d-ecc1-4538-bb04-6710a9cd2078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.096275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ff91d-ecc1-4538-bb04-6710a9cd2078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.098187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ff91d-ecc1-4538-bb04-6710a9cd2078 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.107560 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1380/2772] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:15.368797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.370197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:15.377964 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:15.389097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd060-bda8-4bef-9d9d-a783f99e8297 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.399233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.399897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.400169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.400437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] PATH_INFO: `/v3/projects/82003c14585048029e15b38fc706236d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.401667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82003c14585048029e15b38fc706236d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.403679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.450376 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82003c14585048029e15b38fc706236d', 'request_id': 'req-d6c4f446-be81-4794-9be1-17d18dc79952'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:15.456477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82003c14585048029e15b38fc706236d', 'request_id': 'req-d6c4f446-be81-4794-9be1-17d18dc79952'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:15.464016 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82003c14585048029e15b38fc706236d', 'request_id': 'req-d6c4f446-be81-4794-9be1-17d18dc79952'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:15.475712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] The token cache is being invalidate because project 82003c14585048029e15b38fc706236d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:15.476000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:15.476615 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d6c4f446-be81-4794-9be1-17d18dc79952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:15.478058 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1393/2773] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:15 2022] DELETE /identity/v3/projects/82003c14585048029e15b38fc706236d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:15.483018 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.483322 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.483633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.502524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:15.504245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:15.562453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1381/2774] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:15.570685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.570962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.571236 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.590992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:15.592199 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:15.600062 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1394/2775] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:15.604210 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.613688 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cf4f5-5847-4d37-b099-dc35b9f950d5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.618150 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.618731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.619004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.619271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.619944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 00:58:15.620240 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.621838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.633711 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1382/2776] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 00:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 00:58:15.637870 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.647636 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdb35f-eba4-4ff1-a4e5-e57cc477de15 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.652228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.652823 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.653097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.653364 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.653982 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.655941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.684399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1395/2777] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 00:58:15 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:58:15.688717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.699275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f02077-e248-41ba-90b4-f793f382e9e0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.758032 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.758632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.758937 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.759208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.759878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.761770 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.782849 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1383/2778] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 00:58:15 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 00:58:15.787042 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.797360 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f796-4e63-437b-ae5b-2d28d7f2e7ac admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.802129 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.802719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.802996 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.803267 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.803954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.805564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.815953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1396/2779] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:15.819671 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.829339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e444c9-0079-4c45-b65e-b96a6af16b10 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.834704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93154c7-7816-4b0e-97c1-c48f2d392d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.835288 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93154c7-7816-4b0e-97c1-c48f2d392d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.835602 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93154c7-7816-4b0e-97c1-c48f2d392d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.835875 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93154c7-7816-4b0e-97c1-c48f2d392d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.836511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93154c7-7816-4b0e-97c1-c48f2d392d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.838072 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93154c7-7816-4b0e-97c1-c48f2d392d9a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.848402 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1384/2780] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 00:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 00:58:15.852198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:15.862029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7f375-b023-4099-835f-8f798a8ccaf7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:15.867070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:15.867719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.867997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.868266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] PATH_INFO: `/v3/projects/7655d57f72004c9a8058b615c4cfcd8b/users/937db002d6ba4638ba19834bd32fb277/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.883339 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7655d57f72004c9a8058b615c4cfcd8b, user_id=937db002d6ba4638ba19834bd32fb277, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:15.886033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:15.898524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1397/2781] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 00:58:15 2022] PUT /identity/v3/projects/7655d57f72004c9a8058b615c4cfcd8b/users/937db002d6ba4638ba19834bd32fb277/roles/87f1831257824560813f954348534540 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:15.909624 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:15.909911 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:15.910208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:15.937541 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] MFA Rules not processed for user `937db002d6ba4638ba19834bd32fb277`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:15.938748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:15.969429 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:16.005214 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1385/2782] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:16.016107 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:16.017862 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:16.080386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe1279-25cf-4025-8846-24783865aedc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:16.084217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc104d-04dc-4d2d-9f3f-fce1ba8fce4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:16.084831 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc104d-04dc-4d2d-9f3f-fce1ba8fce4c service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:16.085108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc104d-04dc-4d2d-9f3f-fce1ba8fce4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:16.085378 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc104d-04dc-4d2d-9f3f-fce1ba8fce4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:16.093943 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc104d-04dc-4d2d-9f3f-fce1ba8fce4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:16.095379 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc104d-04dc-4d2d-9f3f-fce1ba8fce4c service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:16.104193 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1398/2783] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:19.325453 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:19.386415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9d4c5-10e4-4216-a654-bf016068bde5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:19.390941 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:19.391583 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:19.391858 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:19.392123 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] PATH_INFO: `/v3/users/937db002d6ba4638ba19834bd32fb277` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:19.393209 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=937db002d6ba4638ba19834bd32fb277)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:19.395083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:19.440273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937db002d6ba4638ba19834bd32fb277'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:19.449504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937db002d6ba4638ba19834bd32fb277'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:19.450147 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937db002d6ba4638ba19834bd32fb277'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:19.456956 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937db002d6ba4638ba19834bd32fb277'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:19.457577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937db002d6ba4638ba19834bd32fb277'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:19.475047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1386/2784] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:58:19 2022] DELETE /identity/v3/users/937db002d6ba4638ba19834bd32fb277 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:19.479606 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52332921-45df-4209-b495-4340df9f489b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:19.480104 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52332921-45df-4209-b495-4340df9f489b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:19.480225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52332921-45df-4209-b495-4340df9f489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:19.501296 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-52332921-45df-4209-b495-4340df9f489b None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:58:19.502513 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-52332921-45df-4209-b495-4340df9f489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:19.520476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:19.562421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1399/2785] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:58:19.572286 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:19.573691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:19.631148 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c5ed-8500-40d7-83f2-794460ecb3ac admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:19.634766 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:19.635357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:19.635662 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:19.635931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:19.641643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:19.646819 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:19.648964 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:19.659337 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1387/2786] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:58:19.925197 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52332921-45df-4209-b495-4340df9f489b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:19.926670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-52332921-45df-4209-b495-4340df9f489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:58:19.939632 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52332921-45df-4209-b495-4340df9f489b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:58:19.944664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:58:19.945292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:58:19.945571 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:58:19.945839 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] PATH_INFO: `/v3/projects/7655d57f72004c9a8058b615c4cfcd8b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:58:19.946993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7655d57f72004c9a8058b615c4cfcd8b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:58:19.949173 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:58:19.996890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7655d57f72004c9a8058b615c4cfcd8b', 'request_id': 'req-9d084c7f-8284-4e89-919d-489a3f481818'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:20.002231 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7655d57f72004c9a8058b615c4cfcd8b', 'request_id': 'req-9d084c7f-8284-4e89-919d-489a3f481818'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:20.010096 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7655d57f72004c9a8058b615c4cfcd8b', 'request_id': 'req-9d084c7f-8284-4e89-919d-489a3f481818'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:20.020889 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] The token cache is being invalidate because project 7655d57f72004c9a8058b615c4cfcd8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:58:20.021711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:20.022319 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:58:20.023802 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1400/2787] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:58:19 2022] DELETE /identity/v3/projects/7655d57f72004c9a8058b615c4cfcd8b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:58:29.655196 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:30.018021 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:58:59.914181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:58:59.915632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:00.000137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27542d-f033-4ff0-ba75-cc014926b3d6 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:00.064437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:00.065052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:00.065330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:00.065611 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:00.067356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:00.126010 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:00.127665 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:00.136913 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1388/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:58:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:59:10.132706 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=84 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:59:12.209063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:12.210659 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:12.269256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d084c7f-8284-4e89-919d-489a3f481818 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:12.272991 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:12.273591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:12.273865 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:12.274133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:12.275945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:12.340836 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:12.342471 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d8e58-1b2e-48d1-a58b-7ea6a0bea497 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:12.352309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1401/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:59:16.233971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:16.236167 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:16.378095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dfb1f-d921-4e2d-a78c-0ce001398e41 service nova] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:16.385596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:16.389794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:16.390112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:16.390381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:16.393580 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:16.415523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:16.417739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:16.431376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1389/2790] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:59:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:59:18.127382 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:18.127926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:18.128049 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:18.155222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] MFA Rules not processed for user `778fc9ee164e46a9b874dd0e20d38392`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 00:59:18.156743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:18.164783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1402/2791] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 00:59:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 00:59:19.704982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:19.717145 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e146-52d2-4ac0-a130-b49550607f6c service placement] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:19.722282 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:19.722876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:19.723153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:19.723423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:19.725273 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:19.737357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:19.738935 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:19.749584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1390/2792] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 00:59:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 00:59:19.779336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:19.780702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:19.795529 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e9091-ba19-47a9-8f8e-ffc8fd9bd5e2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:19.800235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:19.800854 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:19.801131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:19.801425 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] PATH_INFO: `/v3/users/d308262224af475091e6aa494ea839b1` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:19.815521 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d308262224af475091e6aa494ea839b1)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:19.817511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:19.862445 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd308262224af475091e6aa494ea839b1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:19.870691 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd308262224af475091e6aa494ea839b1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:19.871312 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd308262224af475091e6aa494ea839b1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:19.871936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd308262224af475091e6aa494ea839b1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:19.879088 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd308262224af475091e6aa494ea839b1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:19.898120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1403/2793] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:59:19 2022] DELETE /identity/v3/users/d308262224af475091e6aa494ea839b1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:59:20.224353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:20.238563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d631-e361-487a-9b18-423f43c99a7f service placement] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:20.299297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:20.299934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:20.300212 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:20.300483 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] PATH_INFO: `/v3/projects/53d8d76185c44a159439c0f7a0e88b31` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:20.313085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d8d76185c44a159439c0f7a0e88b31)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:20.315957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:20.370933 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d8d76185c44a159439c0f7a0e88b31', 'request_id': 'req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.377517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d8d76185c44a159439c0f7a0e88b31', 'request_id': 'req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.386046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d8d76185c44a159439c0f7a0e88b31', 'request_id': 'req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.394231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:59:20.400413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] The token cache is being invalidate because project 53d8d76185c44a159439c0f7a0e88b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:59:20.400705 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.401555 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.406804 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1391/2794] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:59:20 2022] DELETE /identity/v3/projects/53d8d76185c44a159439c0f7a0e88b31 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:59:20.407856 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:20.409340 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:20.418844 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:59:20.423536 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1112c2-814d-4d32-ba69-e1d44049962c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:20.485256 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:20.485947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:20.486251 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:20.486553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] PATH_INFO: `/v3/users/2822b030129549c09a33b160f5aa6965` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:20.501349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2822b030129549c09a33b160f5aa6965)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:20.503282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:20.550160 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2822b030129549c09a33b160f5aa6965'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.558301 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2822b030129549c09a33b160f5aa6965'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.559143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2822b030129549c09a33b160f5aa6965'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.559798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2822b030129549c09a33b160f5aa6965'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.567640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2822b030129549c09a33b160f5aa6965'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.586618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1404/2795] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 00:59:20 2022] DELETE /identity/v3/users/2822b030129549c09a33b160f5aa6965 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 00:59:20.773561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 00:59:20.775903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 00:59:20.806761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d8e4-4d3d-4bc8-954f-a0ae99667a06 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 00:59:20.871651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 00:59:20.872299 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 00:59:20.872578 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 00:59:20.872848 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] PATH_INFO: `/v3/projects/eb9e29ab32b9438ca3c1680a93d1ac17` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 00:59:20.885748 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9e29ab32b9438ca3c1680a93d1ac17)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 00:59:20.887742 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 00:59:20.950808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9e29ab32b9438ca3c1680a93d1ac17', 'request_id': 'req-6890630f-3fcd-4c9c-a6d7-08883d9424e8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.956703 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9e29ab32b9438ca3c1680a93d1ac17', 'request_id': 'req-6890630f-3fcd-4c9c-a6d7-08883d9424e8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.964921 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9e29ab32b9438ca3c1680a93d1ac17', 'request_id': 'req-6890630f-3fcd-4c9c-a6d7-08883d9424e8'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.978851 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] The token cache is being invalidate because project eb9e29ab32b9438ca3c1680a93d1ac17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 00:59:20.979146 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.980208 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 00:59:20.982169 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1392/2796] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 00:59:20 2022] DELETE /identity/v3/projects/eb9e29ab32b9438ca3c1680a93d1ac17 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 00:59:30.582879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 00:59:30.974616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:36.530913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:36.532357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:36.587433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab7564-a9d6-4cc8-baf1-4feeb2f5607b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:36.638394 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:36.638999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:36.639689 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:36.639972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:36.641707 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:36.711279 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:36.712754 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:36.723427 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1405/2797] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:00:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:00:44.821782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:44.823713 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:44.835638 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890630f-3fcd-4c9c-a6d7-08883d9424e8 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:44.839813 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:44.840396 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:44.840673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:44.840944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:44.842590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:44.855459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:44.857105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:44.868064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1393/2798] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:00:46.718617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=74 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:49.234918 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:49.236409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:49.251131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2cba1-045c-40af-87ed-04e381d4bd08 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:49.257161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4784d64225054ea9a987581a00b6807b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:49.258052 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:49.258326 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:49.258592 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:49.260373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:49.273635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:49.275473 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:49.287424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1406/2799] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:00:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 01:00:54.248223 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:54.249860 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:54.307867 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e70b3f-24a9-41c5-aa96-5708ad2e706d service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:54.311691 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:54.312301 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:54.312582 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:54.312862 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:54.314719 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:54.378078 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:54.379519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:54.389738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1394/2800] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:00:57.472737 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:57.474182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:57.490059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da3ff0-7614-48f2-b1c7-4b5a4bec5742 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:57.494663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:57.495285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:57.495594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:57.496452 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] PATH_INFO: `/v3/users/778fc9ee164e46a9b874dd0e20d38392` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:57.497633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] RBAC: Authorizing `identity:delete_user(user_id=778fc9ee164e46a9b874dd0e20d38392)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:57.499647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:57.547859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '778fc9ee164e46a9b874dd0e20d38392'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.554787 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '778fc9ee164e46a9b874dd0e20d38392'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.555415 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '778fc9ee164e46a9b874dd0e20d38392'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.556046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '778fc9ee164e46a9b874dd0e20d38392'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.563639 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3f9f1dc7-cc80-4b1a-a081-95ae13d8e47c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '778fc9ee164e46a9b874dd0e20d38392'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.580573 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1407/2801] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:00:57 2022] DELETE /identity/v3/users/778fc9ee164e46a9b874dd0e20d38392 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:00:57.754940 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:57.766442 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd9612-bb06-484c-ab74-abb344f5ae03 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:57.823458 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:57.824101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:57.824378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:57.824649 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] PATH_INFO: `/v3/projects/3ef4db17e5d245cb9caab70d557f41ae` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:57.825757 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef4db17e5d245cb9caab70d557f41ae)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:57.827736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:57.866892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:57.874266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef4db17e5d245cb9caab70d557f41ae', 'request_id': 'req-6666adc3-6225-4a41-bcca-a90c32453ab6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.879651 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef4db17e5d245cb9caab70d557f41ae', 'request_id': 'req-6666adc3-6225-4a41-bcca-a90c32453ab6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.886909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef4db17e5d245cb9caab70d557f41ae', 'request_id': 'req-6666adc3-6225-4a41-bcca-a90c32453ab6'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.897765 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] The token cache is being invalidate because project 3ef4db17e5d245cb9caab70d557f41ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:00:57.898051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.898673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-6666adc3-6225-4a41-bcca-a90c32453ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:57.900154 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1395/2802] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:00:57 2022] DELETE /identity/v3/projects/3ef4db17e5d245cb9caab70d557f41ae => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:00:57.905642 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:57.905923 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:57.906190 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:57.926110 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:00:57.927280 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:57.986225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1408/2803] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:00:57.994789 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:57.995075 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:57.995344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.015219 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:00:58.016460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:58.024812 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1396/2804] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:00:58.029477 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.038969 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdf469-47b2-4192-9afb-eb0b2f157c0f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.043523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.044102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.044375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.044641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.045274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 01:00:58.045568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.047145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.058492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1409/2805] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 01:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 01:00:58.062601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.071692 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6649d-1f91-4495-a9eb-1b7c2aa92c3d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.076881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.077403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.077678 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.077945 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.078564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.080473 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.108636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1397/2806] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:00:58 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:00:58.112661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.122539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9370f-6d91-436e-b0e9-ad537e22bf0f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.165304 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:58.181294 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.181909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.182187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.182458 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.183130 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.185134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.210264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1410/2807] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:00:58 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:00:58.211723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.223024 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae475b7-3c76-4b99-a205-1f4c9ddae384 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.227690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.228289 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.228565 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.228836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.229479 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.231134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.248617 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1398/2808] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 01:00:58.249828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.262395 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b8883-bfd5-499e-aaad-118d19c127d3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.267057 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a97a5-2ac2-4223-85cd-fb1e0eda676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.267702 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a97a5-2ac2-4223-85cd-fb1e0eda676a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.267983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a97a5-2ac2-4223-85cd-fb1e0eda676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.268422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a97a5-2ac2-4223-85cd-fb1e0eda676a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.269075 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a97a5-2ac2-4223-85cd-fb1e0eda676a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.270679 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a97a5-2ac2-4223-85cd-fb1e0eda676a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.282474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1411/2809] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:00:58.286551 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.297292 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949168bf-7680-411e-8651-3fdd76473274 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.302168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.302815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.303086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.303351 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] PATH_INFO: `/v3/projects/5e3bec2aa5784363940ba4350bb4bb50/users/437ffe332d82464098092006af14418d/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.318542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3bec2aa5784363940ba4350bb4bb50, user_id=437ffe332d82464098092006af14418d, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.321203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.334256 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1399/2810] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:00:58 2022] PUT /identity/v3/projects/5e3bec2aa5784363940ba4350bb4bb50/users/437ffe332d82464098092006af14418d/roles/87f1831257824560813f954348534540 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:00:58.344447 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.344730 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.344999 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.372409 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] MFA Rules not processed for user `437ffe332d82464098092006af14418d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:00:58.373596 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:58.436441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1412/2811] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:00:58.441053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.501724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787c507-b8e3-441e-acec-7fb8261d04da admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.506312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.506900 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.507174 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.507440 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.508362 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.510231 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.536827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1400/2812] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:00:58 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:00:58.540873 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.551429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c81e6-1c21-4d4d-876c-8027a74f8a64 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.608641 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.609180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.609457 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.609727 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.610358 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.612317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.634461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1413/2813] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:00:58 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:00:58.639084 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.649462 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083735f-de75-4682-a900-c6ac703324b0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.654142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.654732 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.655009 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.655281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.655971 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.657549 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.668027 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1401/2814] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:00:58.672247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.682051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef76ca-5524-4e40-9939-bcb26df9bf32 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.687145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.687811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.688085 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.688354 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] PATH_INFO: `/v3/projects/3ba5b8fbefb24c9f8b36c466b23238f3/users/f53b604f24ff4bb5a0b3dc7288b7c737/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.703118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba5b8fbefb24c9f8b36c466b23238f3, user_id=f53b604f24ff4bb5a0b3dc7288b7c737, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.706014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.718292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1414/2815] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:00:58 2022] PUT /identity/v3/projects/3ba5b8fbefb24c9f8b36c466b23238f3/users/f53b604f24ff4bb5a0b3dc7288b7c737/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:00:58.722347 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.732746 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8148087-9ef0-4cce-99f7-a9effbafd172 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.765600 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:58.789680 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728be2a1-bf96-4c60-87b3-1e4d6c731157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.790276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728be2a1-bf96-4c60-87b3-1e4d6c731157 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.790548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728be2a1-bf96-4c60-87b3-1e4d6c731157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.790816 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728be2a1-bf96-4c60-87b3-1e4d6c731157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.791457 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728be2a1-bf96-4c60-87b3-1e4d6c731157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.793523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728be2a1-bf96-4c60-87b3-1e4d6c731157 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.802931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1402/2816] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:00:58.807386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.818780 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc016d-c3e2-4084-829e-bec4c773aba1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:58.823466 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:58.824180 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.824456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.824724 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] PATH_INFO: `/v3/projects/3ba5b8fbefb24c9f8b36c466b23238f3/users/f53b604f24ff4bb5a0b3dc7288b7c737/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.826595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba5b8fbefb24c9f8b36c466b23238f3, user_id=f53b604f24ff4bb5a0b3dc7288b7c737, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:58.829384 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:58.839238 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1415/2817] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:00:58 2022] PUT /identity/v3/projects/3ba5b8fbefb24c9f8b36c466b23238f3/users/f53b604f24ff4bb5a0b3dc7288b7c737/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:00:58.849492 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:58.849783 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:58.850060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:58.878067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] MFA Rules not processed for user `f53b604f24ff4bb5a0b3dc7288b7c737`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:00:58.879220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:58.946285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1403/2818] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:00:58.957486 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:58.959187 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:58.969046 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:59.018156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392689-0b3d-47c1-9d5f-cc57c1361ee1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:59.021629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d44b6-99e5-4d06-bac1-79078fc9bac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:59.022226 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d44b6-99e5-4d06-bac1-79078fc9bac6 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:59.022506 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d44b6-99e5-4d06-bac1-79078fc9bac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:59.022781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d44b6-99e5-4d06-bac1-79078fc9bac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:59.031713 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d44b6-99e5-4d06-bac1-79078fc9bac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:59.033142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d44b6-99e5-4d06-bac1-79078fc9bac6 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:59.042391 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1416/2819] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:00:58 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:00:59.187127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:59.247393 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64cbc3-959d-4fa7-aa86-8395f8de2f30 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:59.251902 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:59.252517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:59.252793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:59.253064 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] PATH_INFO: `/v3/users/437ffe332d82464098092006af14418d` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:59.254173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=437ffe332d82464098092006af14418d)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:59.256118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:59.303248 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437ffe332d82464098092006af14418d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.310488 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437ffe332d82464098092006af14418d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.311107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437ffe332d82464098092006af14418d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.316527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:00:59.318636 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437ffe332d82464098092006af14418d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.319573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437ffe332d82464098092006af14418d'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.337767 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1404/2820] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:00:59 2022] DELETE /identity/v3/users/437ffe332d82464098092006af14418d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:00:59.342793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:59.343077 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:59.343352 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:59.362843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:00:59.364105 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:59.422846 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1417/2821] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:00:59.432449 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:59.433857 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:59.491276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e694-f69f-4d47-bf92-8a9a0c7b4539 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:59.494883 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:59.495506 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:59.495806 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:59.496074 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:59.505038 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:59.506437 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:59.516410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1405/2822] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:00:59.783980 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:59.785459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:00:59.799419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e53a6b-baba-412d-bb43-dfbc00a05c35 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:59.804414 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:59.805051 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:59.805327 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:59.805658 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] PATH_INFO: `/v3/projects/5e3bec2aa5784363940ba4350bb4bb50` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:59.807121 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3bec2aa5784363940ba4350bb4bb50)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:59.809708 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:00:59.855585 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3bec2aa5784363940ba4350bb4bb50', 'request_id': 'req-b1c69420-bb44-4224-8fc0-e2008844bd1f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.861234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3bec2aa5784363940ba4350bb4bb50', 'request_id': 'req-b1c69420-bb44-4224-8fc0-e2008844bd1f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.868555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3bec2aa5784363940ba4350bb4bb50', 'request_id': 'req-b1c69420-bb44-4224-8fc0-e2008844bd1f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.879838 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] The token cache is being invalidate because project 5e3bec2aa5784363940ba4350bb4bb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:00:59.880125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.880771 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:00:59.882220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1418/2823] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:00:59 2022] DELETE /identity/v3/projects/5e3bec2aa5784363940ba4350bb4bb50 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 01:00:59.886418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:00:59.896089 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ef0fd-a3d7-42c1-a690-d524d3a197c9 service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:00:59.954633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:00:59.955181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:00:59.955461 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:00:59.955762 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] PATH_INFO: `/v3/users/f53b604f24ff4bb5a0b3dc7288b7c737` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:00:59.956894 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53b604f24ff4bb5a0b3dc7288b7c737)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:00:59.958792 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.006217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53b604f24ff4bb5a0b3dc7288b7c737'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.013242 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53b604f24ff4bb5a0b3dc7288b7c737'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.013903 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53b604f24ff4bb5a0b3dc7288b7c737'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.020929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53b604f24ff4bb5a0b3dc7288b7c737'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.021584 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-828ecc17-10e5-48b9-acc7-a2d91ba974e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53b604f24ff4bb5a0b3dc7288b7c737'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.035025 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:00.040142 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1406/2824] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:00:59 2022] DELETE /identity/v3/users/f53b604f24ff4bb5a0b3dc7288b7c737 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:00.304630 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.306146 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:00.320402 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c69420-bb44-4224-8fc0-e2008844bd1f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.351949 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:00.380323 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.380947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.381293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.381633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] PATH_INFO: `/v3/projects/3ba5b8fbefb24c9f8b36c466b23238f3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.382986 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba5b8fbefb24c9f8b36c466b23238f3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.384913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.429922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba5b8fbefb24c9f8b36c466b23238f3', 'request_id': 'req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.435341 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba5b8fbefb24c9f8b36c466b23238f3', 'request_id': 'req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.442392 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba5b8fbefb24c9f8b36c466b23238f3', 'request_id': 'req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.453747 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] The token cache is being invalidate because project 3ba5b8fbefb24c9f8b36c466b23238f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:00.454033 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.454763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4c1b9fe-40fe-40d7-8fe9-5adc613b0571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:00.456670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1419/2825] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:00 2022] DELETE /identity/v3/projects/3ba5b8fbefb24c9f8b36c466b23238f3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:00.461664 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.462080 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.462355 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.482060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:00.483182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:00.540043 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1407/2826] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:00.549169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.549474 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.549743 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.569804 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:00.571000 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:00.579668 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1420/2827] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:00.583736 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.593361 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108e481-6645-49b9-b897-71ae1a1f63e5 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.597922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.598510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.598786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.599060 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.599734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 01:01:00.600034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.601614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.612785 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1408/2828] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 01:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 01:01:00.616886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.626594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cd177-0c08-4a60-9740-14d0956b82cb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.631209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.631828 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.632106 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.632374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.633258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.635185 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.662834 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1421/2829] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:00 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 01:01:00.666895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.676990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed651d-3c75-4125-b097-5d543a378095 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.731631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.732281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.732505 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.732772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.733387 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.735276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.756793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1409/2830] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:00 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:00.761108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.772145 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c418f7-1091-49cb-9ba6-477731f04052 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.776886 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.777475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.777756 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.778029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.778672 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.780316 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.790036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1422/2831] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:00.793843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.803707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764b234-04e9-425e-ba9b-bf72afe70137 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.808826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970070e-20b0-4043-b8f8-ba5e5f293b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.809406 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970070e-20b0-4043-b8f8-ba5e5f293b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.809679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970070e-20b0-4043-b8f8-ba5e5f293b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.809944 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970070e-20b0-4043-b8f8-ba5e5f293b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.810761 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970070e-20b0-4043-b8f8-ba5e5f293b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.812953 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970070e-20b0-4043-b8f8-ba5e5f293b14 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.824372 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1410/2832] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:00.828403 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:00.838309 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba815e6-25d2-48e6-858a-ad2897e674dd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:00.842880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:00.843547 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.843829 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.844094 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] PATH_INFO: `/v3/projects/afe295c8d4e94a19ac7d31f46ee4a36c/users/d1c814fbdce842fdacb2e2cf8cf1f095/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.858091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe295c8d4e94a19ac7d31f46ee4a36c, user_id=d1c814fbdce842fdacb2e2cf8cf1f095, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:00.860775 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:00.873664 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1423/2833] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:00 2022] PUT /identity/v3/projects/afe295c8d4e94a19ac7d31f46ee4a36c/users/d1c814fbdce842fdacb2e2cf8cf1f095/roles/87f1831257824560813f954348534540 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:00.884051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:00.884344 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:00.884672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:00.911968 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] MFA Rules not processed for user `d1c814fbdce842fdacb2e2cf8cf1f095`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:00.913153 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:00.968081 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:00.978129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1411/2834] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:00.983179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.043539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f49b14-669c-4cc0-be65-cde7268ee459 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.048221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.048818 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.049097 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.049373 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.049990 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.051925 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.056338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:01.080946 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1424/2835] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:00 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:01.084957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.095523 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f588e2d-9323-4a04-bb9c-86e0e76c7a78 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.150538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.151129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.151521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.151817 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.152451 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.154343 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.175415 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1412/2836] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:01 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:01.179362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.189555 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728629-fd90-4c87-a95b-b9e658092b62 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.194182 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.194763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.195039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.195308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.195984 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.197572 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.208292 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1425/2837] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:01.212162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.221995 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cbe5b-7b80-467f-a429-22dafb6b1267 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.226533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.227158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.227433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.227734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] PATH_INFO: `/v3/projects/e7f49e5ae9a34f32878a18345e8252f5/users/2d8ae280393e49c9af9ea2aec10ea898/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.242546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f49e5ae9a34f32878a18345e8252f5, user_id=2d8ae280393e49c9af9ea2aec10ea898, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.245144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.258358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1413/2838] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:01 2022] PUT /identity/v3/projects/e7f49e5ae9a34f32878a18345e8252f5/users/2d8ae280393e49c9af9ea2aec10ea898/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:01.262227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.271926 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f62f1-aa74-4724-948a-260aac02e687 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.328953 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a016b3-ffec-47ce-9a40-0be7bedda679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.329557 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a016b3-ffec-47ce-9a40-0be7bedda679 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.329840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a016b3-ffec-47ce-9a40-0be7bedda679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.330116 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a016b3-ffec-47ce-9a40-0be7bedda679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.330793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a016b3-ffec-47ce-9a40-0be7bedda679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.332422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a016b3-ffec-47ce-9a40-0be7bedda679 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.342525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1426/2839] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:01.346682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.357696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43512d-3c9d-41b2-ae2e-205577ee704a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.362376 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.363013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.363290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.363589 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] PATH_INFO: `/v3/projects/e7f49e5ae9a34f32878a18345e8252f5/users/2d8ae280393e49c9af9ea2aec10ea898/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.365504 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f49e5ae9a34f32878a18345e8252f5, user_id=2d8ae280393e49c9af9ea2aec10ea898, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.368168 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.378046 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1414/2840] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:01 2022] PUT /identity/v3/projects/e7f49e5ae9a34f32878a18345e8252f5/users/2d8ae280393e49c9af9ea2aec10ea898/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:01.389162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.389450 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.389728 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.419424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] MFA Rules not processed for user `2d8ae280393e49c9af9ea2aec10ea898`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:01.420694 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:01.486545 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1427/2841] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:01.497932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.499604 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:01.559034 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570abb83-4e1f-49f3-8ddb-9ec5bab9d711 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.562723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.563353 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.563656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.563925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.572982 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.574420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.584085 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1415/2842] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:01.660274 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:01.671072 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f9ad9-f78f-4f60-98ce-7a75982e6036 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:01.674669 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410413-69bd-4894-8dd2-72329716e301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:01.675580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410413-69bd-4894-8dd2-72329716e301 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:01.675580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410413-69bd-4894-8dd2-72329716e301 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:01.675798 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410413-69bd-4894-8dd2-72329716e301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:01.729209 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:01.734008 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99410413-69bd-4894-8dd2-72329716e301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:01.735887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99410413-69bd-4894-8dd2-72329716e301 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:01.744755 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1428/2843] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:11.579621 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=72,keystone:INSERT=5 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:11.740687 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:13.053637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:13.112739 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a1ffb-6055-44e3-950b-ca17c1de7da3 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:13.117218 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:13.117820 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:13.118093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:13.118357 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] PATH_INFO: `/v3/users/d1c814fbdce842fdacb2e2cf8cf1f095` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:13.119454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c814fbdce842fdacb2e2cf8cf1f095)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:13.121413 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:13.165707 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c814fbdce842fdacb2e2cf8cf1f095'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.172460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c814fbdce842fdacb2e2cf8cf1f095'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.173094 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c814fbdce842fdacb2e2cf8cf1f095'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.180590 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c814fbdce842fdacb2e2cf8cf1f095'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.181162 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c814fbdce842fdacb2e2cf8cf1f095'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.197920 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1416/2844] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:13 2022] DELETE /identity/v3/users/d1c814fbdce842fdacb2e2cf8cf1f095 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:13.202970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:13.203446 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:13.203581 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:13.222625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:13.223827 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:13.282629 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1429/2845] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:13.292544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:13.293973 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:13.349217 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4b753-17e0-462a-a4af-b8e37df562c6 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:13.352842 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:13.353431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:13.353710 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:13.353988 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:13.362731 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:13.364163 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:13.373870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1417/2846] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:13.654043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:13.655503 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:13.678211 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ddb3b-47f9-464a-9ed8-950f728d4e14 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:13.684170 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:13.684797 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:13.685070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:13.685336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] PATH_INFO: `/v3/projects/afe295c8d4e94a19ac7d31f46ee4a36c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:13.686442 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe295c8d4e94a19ac7d31f46ee4a36c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:13.688389 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:13.733897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe295c8d4e94a19ac7d31f46ee4a36c', 'request_id': 'req-13a541b2-8bc1-46d4-a341-d4a97b67d091'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.739118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe295c8d4e94a19ac7d31f46ee4a36c', 'request_id': 'req-13a541b2-8bc1-46d4-a341-d4a97b67d091'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.748779 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe295c8d4e94a19ac7d31f46ee4a36c', 'request_id': 'req-13a541b2-8bc1-46d4-a341-d4a97b67d091'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.760258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] The token cache is being invalidate because project afe295c8d4e94a19ac7d31f46ee4a36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:13.760566 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.761204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.762654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1430/2847] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:13 2022] DELETE /identity/v3/projects/afe295c8d4e94a19ac7d31f46ee4a36c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:13.766644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:13.776200 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20146c39-7832-48bc-a19d-eef35e62a7fe service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:13.794040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:13.833538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:13.834158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:13.834439 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:13.834709 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] PATH_INFO: `/v3/users/2d8ae280393e49c9af9ea2aec10ea898` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:13.835859 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d8ae280393e49c9af9ea2aec10ea898)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:13.837796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:13.882053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d8ae280393e49c9af9ea2aec10ea898'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.889598 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d8ae280393e49c9af9ea2aec10ea898'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.890266 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d8ae280393e49c9af9ea2aec10ea898'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.897114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d8ae280393e49c9af9ea2aec10ea898'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.897753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-d588704c-3501-42d9-879d-213ac606540b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d8ae280393e49c9af9ea2aec10ea898'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:13.914133 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1418/2848] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:13 2022] DELETE /identity/v3/users/2d8ae280393e49c9af9ea2aec10ea898 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:14.182317 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.183888 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:14.197118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a541b2-8bc1-46d4-a341-d4a97b67d091 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.255385 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.256138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.256308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.256580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] PATH_INFO: `/v3/projects/e7f49e5ae9a34f32878a18345e8252f5` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.257951 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f49e5ae9a34f32878a18345e8252f5)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.259917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.289455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:14.306763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f49e5ae9a34f32878a18345e8252f5', 'request_id': 'req-4815bfc4-c369-497a-b84d-c3b138123cf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:14.312265 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f49e5ae9a34f32878a18345e8252f5', 'request_id': 'req-4815bfc4-c369-497a-b84d-c3b138123cf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:14.319419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f49e5ae9a34f32878a18345e8252f5', 'request_id': 'req-4815bfc4-c369-497a-b84d-c3b138123cf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:14.330420 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] The token cache is being invalidate because project e7f49e5ae9a34f32878a18345e8252f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:14.330706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:14.331375 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4815bfc4-c369-497a-b84d-c3b138123cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:14.332899 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1431/2849] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:14 2022] DELETE /identity/v3/projects/e7f49e5ae9a34f32878a18345e8252f5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:14.337923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.338397 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.338514 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.358126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:14.359330 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:14.416948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1419/2850] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:14.424568 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.424849 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.425120 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.444487 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:14.445667 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:14.454300 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1432/2851] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:14.458603 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.468278 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764cad-637e-4fae-a2c1-a1e12d4743fa None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.472827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.473447 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.473718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.473986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.474626 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81609) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 01:01:14.474926 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.476595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.488296 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1420/2852] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 01:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 01:01:14.492608 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.502386 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2814bab-0778-4c87-aa46-d0a85a3ce495 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.507006 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.507625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.508157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.508459 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.509078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.510979 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.537424 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1433/2853] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:14.541672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.551958 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea38ed2-b19f-4e38-99b8-0e740c421fdf admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.577272 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:14.607763 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.608360 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.608637 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.608907 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.609535 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.611421 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.632434 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1421/2854] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 01:01:14.636736 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.646965 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94287-14c4-426e-8a84-c5757f209145 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.651837 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.652435 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.652714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.652983 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.653625 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.655227 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.665161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1434/2855] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:14.669828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.680013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d349e-a401-4602-ae67-389ae3c04df1 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.685696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a07a9-1541-4670-b7f8-93fc6b59cfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.686290 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a07a9-1541-4670-b7f8-93fc6b59cfcb admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.686568 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a07a9-1541-4670-b7f8-93fc6b59cfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.686838 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a07a9-1541-4670-b7f8-93fc6b59cfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.687644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a07a9-1541-4670-b7f8-93fc6b59cfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.689245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a07a9-1541-4670-b7f8-93fc6b59cfcb admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.699843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1422/2856] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:14.703885 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.713748 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81051f6-39e4-4081-b9ba-f08a6a5ca4a1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.718268 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.718906 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.719186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.719456 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] PATH_INFO: `/v3/projects/6af43b5018a04ba9adcaf70b70e4b1fe/users/03a2e42267bd421da07fbc5ca6c30a15/roles/87f1831257824560813f954348534540` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.734480 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af43b5018a04ba9adcaf70b70e4b1fe, user_id=03a2e42267bd421da07fbc5ca6c30a15, role_id=87f1831257824560813f954348534540)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.737183 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.751587 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1435/2857] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:14 2022] PUT /identity/v3/projects/6af43b5018a04ba9adcaf70b70e4b1fe/users/03a2e42267bd421da07fbc5ca6c30a15/roles/87f1831257824560813f954348534540 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:14.761808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.762095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.762367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.789548 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] MFA Rules not processed for user `03a2e42267bd421da07fbc5ca6c30a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:14.790764 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:14.856067 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1423/2858] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:14.860603 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.920987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773e39b-1bd2-4eac-9e80-f7c984a53866 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:14.925530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:14.926143 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:14.926416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:14.926683 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:14.927310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:14.929218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:14.956074 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1436/2859] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:14.960216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:14.970631 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d1e2e-b0b6-45d2-928d-6df342961f1f None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.025202 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.025795 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.026068 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.026332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] PATH_INFO: `/v3/users` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.026979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.028889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.049020 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1424/2860] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:14 2022] POST /identity/v3/users => generated 299 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:15.053136 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.063897 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52311e7-6feb-4482-bc1a-08f1626bf5a7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.069125 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.070015 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.070225 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.070507 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.071157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.072788 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.084012 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1437/2861] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:15.088139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.097960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1a6b-e68f-4a24-8bcc-bc646d67fb51 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.102495 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.103127 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.103400 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.103698 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] PATH_INFO: `/v3/projects/b5c07877eeae4ca68cc943b25f372c6d/users/0a537f9e6785437f944d5fc24ecc2979/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.118005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c07877eeae4ca68cc943b25f372c6d, user_id=0a537f9e6785437f944d5fc24ecc2979, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.121115 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.133489 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1425/2862] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:15 2022] PUT /identity/v3/projects/b5c07877eeae4ca68cc943b25f372c6d/users/0a537f9e6785437f944d5fc24ecc2979/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:15.137404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.147131 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159124cc-6569-4eb6-9aea-fe87d0aa1569 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.166685 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:15.204431 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170865f4-b644-430d-8223-edd56b9fdabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.205029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170865f4-b644-430d-8223-edd56b9fdabc admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.205313 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170865f4-b644-430d-8223-edd56b9fdabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.205588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170865f4-b644-430d-8223-edd56b9fdabc admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.206232 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170865f4-b644-430d-8223-edd56b9fdabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.207867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170865f4-b644-430d-8223-edd56b9fdabc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.217936 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1438/2863] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:15.222111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.227438 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:15.233704 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf8686-2c59-4278-9f54-945ac9e87985 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.238744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.239554 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.239835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.240106 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] PATH_INFO: `/v3/projects/b5c07877eeae4ca68cc943b25f372c6d/users/0a537f9e6785437f944d5fc24ecc2979/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.242015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c07877eeae4ca68cc943b25f372c6d, user_id=0a537f9e6785437f944d5fc24ecc2979, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.244687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.254837 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1426/2864] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:15 2022] PUT /identity/v3/projects/b5c07877eeae4ca68cc943b25f372c6d/users/0a537f9e6785437f944d5fc24ecc2979/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:15.267020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.267527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.267654 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.296405 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] MFA Rules not processed for user `0a537f9e6785437f944d5fc24ecc2979`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:15.297647 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:15.363553 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1439/2865] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:15.375454 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.377230 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:15.433676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d734e30-9294-4f01-bf82-fcfb564bf970 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.437331 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e644b-d78e-4d8b-8dcf-a096edcbc671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.437916 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e644b-d78e-4d8b-8dcf-a096edcbc671 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.438189 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e644b-d78e-4d8b-8dcf-a096edcbc671 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.438456 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e644b-d78e-4d8b-8dcf-a096edcbc671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.447138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e644b-d78e-4d8b-8dcf-a096edcbc671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.449088 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e644b-d78e-4d8b-8dcf-a096edcbc671 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.458294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1427/2866] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:15.523564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.584138 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa87fd8-19bc-495a-8c5f-f1ecd68c43df None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.588670 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.589783 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.590053 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.590318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] PATH_INFO: `/v3/users/03a2e42267bd421da07fbc5ca6c30a15` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.591422 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a2e42267bd421da07fbc5ca6c30a15)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.593372 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.638580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a2e42267bd421da07fbc5ca6c30a15'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:15.640159 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:15.646921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a2e42267bd421da07fbc5ca6c30a15'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:15.647719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a2e42267bd421da07fbc5ca6c30a15'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:15.648515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a2e42267bd421da07fbc5ca6c30a15'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:15.656179 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a2e42267bd421da07fbc5ca6c30a15'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:15.673448 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1440/2867] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:15 2022] DELETE /identity/v3/users/03a2e42267bd421da07fbc5ca6c30a15 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:15.677987 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.678276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.678571 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.697316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:15.698510 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:15.758744 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1428/2868] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:15.768484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:15.769872 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:15.826020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b527b02-c770-411a-8f27-fc91b4726ea4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:15.829579 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:15.830171 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:15.830444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:15.830711 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:15.839947 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:15.841371 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:15.850444 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1441/2869] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:16.113297 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:16.114852 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:16.127139 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85dd88-c8da-4e75-88d4-a7371e0d5011 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:16.131882 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:16.132515 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.133098 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.133367 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] PATH_INFO: `/v3/projects/6af43b5018a04ba9adcaf70b70e4b1fe` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.134511 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af43b5018a04ba9adcaf70b70e4b1fe)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:16.136986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:16.181855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af43b5018a04ba9adcaf70b70e4b1fe', 'request_id': 'req-8d14ea85-5583-4f1d-af9f-ab8addf60653'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.187181 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af43b5018a04ba9adcaf70b70e4b1fe', 'request_id': 'req-8d14ea85-5583-4f1d-af9f-ab8addf60653'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.194542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af43b5018a04ba9adcaf70b70e4b1fe', 'request_id': 'req-8d14ea85-5583-4f1d-af9f-ab8addf60653'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.205990 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] The token cache is being invalidate because project 6af43b5018a04ba9adcaf70b70e4b1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:16.206275 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.206922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.208417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1429/2870] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:16 2022] DELETE /identity/v3/projects/6af43b5018a04ba9adcaf70b70e4b1fe => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:16.212492 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:16.223032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4623f76-767a-4501-b507-3dddcb3dc4c1 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:16.282440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:16.283134 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.283356 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.283661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] PATH_INFO: `/v3/users/0a537f9e6785437f944d5fc24ecc2979` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.284792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a537f9e6785437f944d5fc24ecc2979)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:16.286693 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:16.333911 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a537f9e6785437f944d5fc24ecc2979'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.341390 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a537f9e6785437f944d5fc24ecc2979'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.341997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a537f9e6785437f944d5fc24ecc2979'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.342595 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a537f9e6785437f944d5fc24ecc2979'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.349618 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c860246c-8bdd-4617-85f9-5ced128990b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a537f9e6785437f944d5fc24ecc2979'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.354726 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:16.368191 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1442/2871] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:16 2022] DELETE /identity/v3/users/0a537f9e6785437f944d5fc24ecc2979 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:16.637895 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:16.639366 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:16.659281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14ea85-5583-4f1d-af9f-ab8addf60653 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:16.671919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:16.717774 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:16.718325 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.718599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.718866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] PATH_INFO: `/v3/projects/b5c07877eeae4ca68cc943b25f372c6d` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.719998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c07877eeae4ca68cc943b25f372c6d)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:16.721878 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:16.767112 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c07877eeae4ca68cc943b25f372c6d', 'request_id': 'req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.772545 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c07877eeae4ca68cc943b25f372c6d', 'request_id': 'req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.779699 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c07877eeae4ca68cc943b25f372c6d', 'request_id': 'req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.790654 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] The token cache is being invalidate because project b5c07877eeae4ca68cc943b25f372c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:16.790938 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.791591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-0c4ab4f4-96cb-47a2-9a59-21b9c3569554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:16.793643 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1430/2872] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:16 2022] DELETE /identity/v3/projects/b5c07877eeae4ca68cc943b25f372c6d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:16.798530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.798814 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.799087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.818998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:16.820215 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:16.878338 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1443/2873] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:16.886391 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.886672 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.886948 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.906750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:16.908015 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:16.916423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1431/2874] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:16.921602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:16.931714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da93574-6747-4122-bdfe-8af8ef1ed13c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:16.936954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:16.937550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.937830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.938103 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.938913 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 01:01:16.939439 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:16.941484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:16.953175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1444/2875] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 01:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 01:01:16.957642 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:16.966864 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280523b-8653-4403-920c-4c6c9d4422f0 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:16.971564 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:16.972100 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:16.972379 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:16.972648 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:16.973281 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:16.975158 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.002144 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1432/2876] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:16 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:17.006220 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.016370 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598f68-9a63-4194-8eb1-9e2169fde539 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.071060 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.071720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.071998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.072266 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.072883 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.074867 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.096203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1445/2877] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:17 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:17.100356 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.110587 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12788638-b97c-4062-855f-90a1c9d0cebf admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.115221 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.115853 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.116130 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.116403 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.117040 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.118612 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.128966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1433/2878] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:17.133221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.143100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c1ffc-918d-4acb-844b-0162177544de admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.147720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949947ed-9961-4424-aed2-ec4e80440054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.148314 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949947ed-9961-4424-aed2-ec4e80440054 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.148594 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949947ed-9961-4424-aed2-ec4e80440054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.148868 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949947ed-9961-4424-aed2-ec4e80440054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.149511 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949947ed-9961-4424-aed2-ec4e80440054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.151127 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949947ed-9961-4424-aed2-ec4e80440054 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.161987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1446/2879] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:17.166011 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.176368 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af3e33-c0b6-4580-b15f-914e17d0917f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.180991 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.181620 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.181899 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.182170 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] PATH_INFO: `/v3/projects/f2cd1d7009a94b15a3185daf4d178d43/users/4f48a2ea39b647aca8ed13180f8ae5c4/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.197405 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2cd1d7009a94b15a3185daf4d178d43, user_id=4f48a2ea39b647aca8ed13180f8ae5c4, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.200108 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.212597 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1434/2880] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:17 2022] PUT /identity/v3/projects/f2cd1d7009a94b15a3185daf4d178d43/users/4f48a2ea39b647aca8ed13180f8ae5c4/roles/87f1831257824560813f954348534540 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:17.223255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.223651 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.223972 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.251578 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] MFA Rules not processed for user `4f48a2ea39b647aca8ed13180f8ae5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:17.252772 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:17.294463 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:17.318336 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1447/2881] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:17.322942 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.366866 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:17.385498 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8373b1-c8ad-413c-8683-2fd61737e988 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.390245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.390835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.391111 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.391378 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.392052 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.393931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.420596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1435/2882] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:17 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:17.424739 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.435247 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe24d5-f326-4836-8d9d-2cc727942eed None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.491534 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.492071 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.492349 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.492617 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.493282 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.495222 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.517325 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1448/2883] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:17 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:17.521245 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.532051 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db690b4-9a5f-4d20-955d-27a574885572 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.536629 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.537220 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.537502 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.537777 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.538420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.540044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.550395 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1436/2884] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 01:01:17.554410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.564295 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685b8c7-a49e-4ac2-948b-54aec36d0460 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.568887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.569524 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.569803 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.570080 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] PATH_INFO: `/v3/projects/f518b28c50204fb9b02d5896c84fcb7c/users/8425252676c14e95b74639342c12fee7/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.584935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f518b28c50204fb9b02d5896c84fcb7c, user_id=8425252676c14e95b74639342c12fee7, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.587627 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.600941 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1449/2885] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:17 2022] PUT /identity/v3/projects/f518b28c50204fb9b02d5896c84fcb7c/users/8425252676c14e95b74639342c12fee7/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 01:01:17.604793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.614538 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b67c06-354f-43ee-a39c-dbc3110faf42 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.670957 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b749c3-2687-4128-ba0c-9d0ed960759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.671635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b749c3-2687-4128-ba0c-9d0ed960759f admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.671854 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b749c3-2687-4128-ba0c-9d0ed960759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.672116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b749c3-2687-4128-ba0c-9d0ed960759f admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.672747 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b749c3-2687-4128-ba0c-9d0ed960759f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.674323 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b749c3-2687-4128-ba0c-9d0ed960759f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.684701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1437/2886] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:17.688550 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.700034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa54e-ff4f-42e5-9540-3e08c0ff2cae admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.704663 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.705291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.705565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.705832 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] PATH_INFO: `/v3/projects/f518b28c50204fb9b02d5896c84fcb7c/users/8425252676c14e95b74639342c12fee7/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.708514 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f518b28c50204fb9b02d5896c84fcb7c, user_id=8425252676c14e95b74639342c12fee7, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.711344 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.723589 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1450/2887] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:17 2022] PUT /identity/v3/projects/f518b28c50204fb9b02d5896c84fcb7c/users/8425252676c14e95b74639342c12fee7/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:17.735198 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.735497 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.735784 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.763810 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] MFA Rules not processed for user `8425252676c14e95b74639342c12fee7`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:17.764965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:17.836827 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1438/2888] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:17.847602 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.849293 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:17.906348 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfc24e-8be2-47b0-88ce-7cefc35fae78 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.910019 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.910610 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.910891 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.911162 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:17.920007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:17.921433 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:17.930635 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1451/2889] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:17.979681 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:17.981095 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:17.992977 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accf0ec-0d52-4c7f-a8b8-079e30e1055d None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:17.997184 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ffd0f-2d70-4fe1-87ba-6de9061e0545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:17.997775 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ffd0f-2d70-4fe1-87ba-6de9061e0545 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:17.998053 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ffd0f-2d70-4fe1-87ba-6de9061e0545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:17.998324 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ffd0f-2d70-4fe1-87ba-6de9061e0545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:18.031423 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:18.054965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ffd0f-2d70-4fe1-87ba-6de9061e0545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:18.056418 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ffd0f-2d70-4fe1-87ba-6de9061e0545 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:18.065635 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1439/2890] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:21.324374 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:21.364609 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:21.387954 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ddf1-2ec8-4cf9-bc70-a3bc1b456720 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:21.392607 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:21.393221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:21.393495 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:21.393763 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] PATH_INFO: `/v3/users/4f48a2ea39b647aca8ed13180f8ae5c4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:21.394857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f48a2ea39b647aca8ed13180f8ae5c4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:21.396793 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:21.445218 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f48a2ea39b647aca8ed13180f8ae5c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:21.452169 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f48a2ea39b647aca8ed13180f8ae5c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:21.452880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f48a2ea39b647aca8ed13180f8ae5c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:21.453484 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f48a2ea39b647aca8ed13180f8ae5c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:21.460896 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f48a2ea39b647aca8ed13180f8ae5c4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:21.477499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1452/2891] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:21 2022] DELETE /identity/v3/users/4f48a2ea39b647aca8ed13180f8ae5c4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:21.482509 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:21.483039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:21.483039 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:21.502086 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:21.503294 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:21.560121 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1440/2892] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:21.569874 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:21.571287 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:21.627189 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460387e-ceaf-496d-bdb2-9d2c8464d8ad admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:21.630811 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:21.631404 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:21.631715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:21.631993 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:21.641653 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:21.643091 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:21.652660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1453/2893] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:21.917738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:21.919280 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:21.931896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c8897-683d-4366-8dc2-725bd2e6f6da None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:21.936517 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:21.937141 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:21.937417 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:21.937687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] PATH_INFO: `/v3/projects/f2cd1d7009a94b15a3185daf4d178d43` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:21.938794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2cd1d7009a94b15a3185daf4d178d43)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:21.940737 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:21.987420 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2cd1d7009a94b15a3185daf4d178d43', 'request_id': 'req-8118eb7d-b4df-4c6d-bd26-5cde7244f963'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:21.993023 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2cd1d7009a94b15a3185daf4d178d43', 'request_id': 'req-8118eb7d-b4df-4c6d-bd26-5cde7244f963'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.000109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2cd1d7009a94b15a3185daf4d178d43', 'request_id': 'req-8118eb7d-b4df-4c6d-bd26-5cde7244f963'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.011599 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] The token cache is being invalidate because project f2cd1d7009a94b15a3185daf4d178d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:22.011887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.012519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.013983 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1441/2894] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:21 2022] DELETE /identity/v3/projects/f2cd1d7009a94b15a3185daf4d178d43 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:22.017904 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.028235 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a052374-5535-418c-bc57-aad479ecdb59 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.081890 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.082515 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.082796 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.083070 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] PATH_INFO: `/v3/users/8425252676c14e95b74639342c12fee7` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.084250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8425252676c14e95b74639342c12fee7)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.086278 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.087935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:22.134465 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8425252676c14e95b74639342c12fee7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.141937 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8425252676c14e95b74639342c12fee7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.142706 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8425252676c14e95b74639342c12fee7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.143302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8425252676c14e95b74639342c12fee7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.150109 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-021c0cb2-4409-4148-b3b3-5dfedc83b1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8425252676c14e95b74639342c12fee7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.167302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1454/2895] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:22 2022] DELETE /identity/v3/users/8425252676c14e95b74639342c12fee7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:22.434986 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.436656 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:22.454216 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118eb7d-b4df-4c6d-bd26-5cde7244f963 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.512546 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.513178 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.513359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.513628 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] PATH_INFO: `/v3/projects/f518b28c50204fb9b02d5896c84fcb7c` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.514738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f518b28c50204fb9b02d5896c84fcb7c)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.516640 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.518045 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:22.565915 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f518b28c50204fb9b02d5896c84fcb7c', 'request_id': 'req-43aed14d-0270-4c3f-9461-1538e7c014ac'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.571276 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f518b28c50204fb9b02d5896c84fcb7c', 'request_id': 'req-43aed14d-0270-4c3f-9461-1538e7c014ac'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.579876 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f518b28c50204fb9b02d5896c84fcb7c', 'request_id': 'req-43aed14d-0270-4c3f-9461-1538e7c014ac'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.590738 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] The token cache is being invalidate because project f518b28c50204fb9b02d5896c84fcb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:22.591062 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.591723 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-43aed14d-0270-4c3f-9461-1538e7c014ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:22.593206 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1442/2896] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:22 2022] DELETE /identity/v3/projects/f518b28c50204fb9b02d5896c84fcb7c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:22.598079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.598561 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.598673 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.618100 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:22.618705 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:22.687307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1455/2897] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 01:01:22.697716 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.698786 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.699073 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.724579 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:22.726309 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:22.735797 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1443/2898] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:22.740114 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.749742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b66238-cbda-41e4-a845-3868ff47bfde None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.754310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.754910 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.755186 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.755455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.756133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 01:01:22.756436 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.758285 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.770362 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1456/2899] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 01:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 01:01:22.774577 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.784369 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9639d3-2c6c-4ec7-b4de-c6f4ecd01fbc None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.788979 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.789561 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.789835 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.790101 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.790729 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.792676 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.818392 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1444/2900] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:22 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 01:01:22.822334 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.833228 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79152382-0038-4c98-accb-762e6c58b8f4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.889142 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.889742 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.890020 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.890291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.890921 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.892859 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.904258 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:22.915824 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1457/2901] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:22 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:22.919796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.930316 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7d17e-bd81-485f-8504-7f682cd0342f admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.937595 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.938203 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.939690 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.939970 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.940644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.942235 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.952874 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1445/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:22.956800 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.967254 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80637f-e54c-4577-a499-cf699a2d88e5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:22.971957 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aad23-2448-4bb9-b4e5-76db533091a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:22.972539 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aad23-2448-4bb9-b4e5-76db533091a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:22.972813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aad23-2448-4bb9-b4e5-76db533091a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:22.973079 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aad23-2448-4bb9-b4e5-76db533091a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:22.973712 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aad23-2448-4bb9-b4e5-76db533091a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:22.975289 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aad23-2448-4bb9-b4e5-76db533091a2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:22.985805 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1458/2903] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:22.989897 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:22.999596 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539d58-d46f-4756-9086-930f3bdafad2 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:23.004126 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:23.004782 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:23.005057 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:23.005320 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] PATH_INFO: `/v3/projects/2cbbe2036d224203b013e4182754e172/users/e5c569581e24484f9a394db98e126bf9/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:23.020044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbbe2036d224203b013e4182754e172, user_id=e5c569581e24484f9a394db98e126bf9, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:23.022673 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:23.036118 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1446/2904] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:22 2022] PUT /identity/v3/projects/2cbbe2036d224203b013e4182754e172/users/e5c569581e24484f9a394db98e126bf9/roles/87f1831257824560813f954348534540 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:23.048892 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:23.049175 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:23.049475 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:23.077998 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] MFA Rules not processed for user `e5c569581e24484f9a394db98e126bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:23.079210 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:23.141548 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1459/2905] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:23.152934 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:23.154601 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:23.230128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16f75c-7a84-4831-9594-9bf1a679fad0 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:23.233724 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a3782-1f1c-4728-be95-2da6e572abc7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2cf22448884e1f8758b9a448d001ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:23.234311 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3782-1f1c-4728-be95-2da6e572abc7 service glance] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:23.234586 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3782-1f1c-4728-be95-2da6e572abc7 service glance] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:23.234855 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3782-1f1c-4728-be95-2da6e572abc7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:23.242905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a3782-1f1c-4728-be95-2da6e572abc7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:23.244345 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a3782-1f1c-4728-be95-2da6e572abc7 service glance] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:23.253342 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1447/2906] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:33.138616 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=39 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:33.249448 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:45.077807 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:45.140233 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8de858-0e6b-4531-85ca-020dc0b6a454 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:45.144792 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:45.145416 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.145696 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.145970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] PATH_INFO: `/v3/users/e5c569581e24484f9a394db98e126bf9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.147089 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c569581e24484f9a394db98e126bf9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:45.149047 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:45.195622 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c569581e24484f9a394db98e126bf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.203719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c569581e24484f9a394db98e126bf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.204337 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c569581e24484f9a394db98e126bf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.204962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c569581e24484f9a394db98e126bf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.211864 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c569581e24484f9a394db98e126bf9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.229879 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1460/2907] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:45 2022] DELETE /identity/v3/users/e5c569581e24484f9a394db98e126bf9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:45.234553 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.235031 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.235137 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.254093 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:45.255291 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:45.311962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1448/2908] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:45.321527 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:45.323419 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:45.379032 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8c7a9-3f24-4935-8b4f-3cf5720dbe78 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:45.382746 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317853f-8bb7-45af-a7e2-b29b835a3d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:45.383633 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317853f-8bb7-45af-a7e2-b29b835a3d54 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.383909 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317853f-8bb7-45af-a7e2-b29b835a3d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.384177 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317853f-8bb7-45af-a7e2-b29b835a3d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.393400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317853f-8bb7-45af-a7e2-b29b835a3d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:45.394817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317853f-8bb7-45af-a7e2-b29b835a3d54 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:45.404604 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1461/2909] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:45.667931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:45.673544 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:45.682889 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340a29-e058-4aa1-afe1-33950d617c80 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:45.687563 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:45.688195 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.688466 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.688730 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] PATH_INFO: `/v3/projects/2cbbe2036d224203b013e4182754e172` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.689836 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbbe2036d224203b013e4182754e172)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:45.692558 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:45.740994 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbbe2036d224203b013e4182754e172', 'request_id': 'req-71c1b19b-5271-4b6a-9c6e-f2467413ae96'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.747087 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbbe2036d224203b013e4182754e172', 'request_id': 'req-71c1b19b-5271-4b6a-9c6e-f2467413ae96'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.754519 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbbe2036d224203b013e4182754e172', 'request_id': 'req-71c1b19b-5271-4b6a-9c6e-f2467413ae96'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.765890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] The token cache is being invalidate because project 2cbbe2036d224203b013e4182754e172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81609) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:45.766182 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.766833 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-71c1b19b-5271-4b6a-9c6e-f2467413ae96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:45.768419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1449/2910] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:45 2022] DELETE /identity/v3/projects/2cbbe2036d224203b013e4182754e172 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:45.773264 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.773576 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.773855 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.794108 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:45.795308 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:45.811799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:45.854790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1462/2911] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:45.862591 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.862847 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.863143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.882499 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:45.884000 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:45.891887 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1450/2912] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:45.896819 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:45.906857 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c959-4cec-4675-97a7-d11f289a299d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:45.911357 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:45.911977 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.912250 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.912520 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.913161 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 01:01:45.913461 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:45.915048 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:45.926938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1463/2913] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 01:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 01:01:45.930966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:45.940114 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f5821-7c80-4fff-ac85-634a6902f5f9 None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:45.944750 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:45.945333 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:45.945613 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:45.945884 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:45.946507 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:45.948965 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:45.975925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1451/2914] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:45 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:45.980034 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:45.989699 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f2f08-deaf-445b-8871-c8f07c25e884 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.043840 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.044421 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.044695 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.044962 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.045580 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.047467 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.068283 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1464/2915] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:45 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 01:01:46.072321 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.082898 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c27f1a-dc36-4fd6-9c2b-f02c0af3cd4e admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.087428 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.088042 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.088909 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.089187 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.089822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.091412 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.101446 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1452/2916] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 01:01:46.105307 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.114971 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48289666-4325-446d-aa9d-21dc338e9e34 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.119661 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26158a53-dd8a-4184-923b-4bcffde55586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.120242 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26158a53-dd8a-4184-923b-4bcffde55586 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.120519 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26158a53-dd8a-4184-923b-4bcffde55586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.120790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26158a53-dd8a-4184-923b-4bcffde55586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.121429 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26158a53-dd8a-4184-923b-4bcffde55586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.123014 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26158a53-dd8a-4184-923b-4bcffde55586 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.133605 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1465/2917] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:46.138058 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.147989 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249437d1-62ba-4441-b6ee-3641207246b7 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.153173 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.153808 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.154083 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.154359 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] PATH_INFO: `/v3/projects/860fd9e3759345a18a6e1699bd9b4279/users/86a4f6d6685b47fab6816b4fa5d4b519/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.168794 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860fd9e3759345a18a6e1699bd9b4279, user_id=86a4f6d6685b47fab6816b4fa5d4b519, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.171444 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.184257 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1453/2918] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:46 2022] PUT /identity/v3/projects/860fd9e3759345a18a6e1699bd9b4279/users/86a4f6d6685b47fab6816b4fa5d4b519/roles/87f1831257824560813f954348534540 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:46.194577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.194863 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.195157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.222997 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] MFA Rules not processed for user `86a4f6d6685b47fab6816b4fa5d4b519`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:46.224221 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:46.289202 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1466/2919] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 01:01:46.293701 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.314799 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:46.355682 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64fea7-764a-4736-a907-460896709a96 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.360232 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.360828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.361109 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.361381 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.362008 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.363922 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.390271 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1454/2920] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:46 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:46.394469 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.405437 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790dc24-ea8a-44a7-b3d8-8e7e550e074c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.417564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:46.462929 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.463660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.463938 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.464207 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.464830 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.466720 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.486588 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1467/2921] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:46 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 01:01:46.490570 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.501161 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8af8f5-cc89-4795-ad5a-c7bd884bd043 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.505679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.506259 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.506533 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.506800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.507433 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.509285 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.521113 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1455/2922] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 01:01:46.524851 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.534657 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d6203-6395-4b38-af44-2fe4d75e2adb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.539234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846623a-3db4-4e22-994c-5f237944363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.539847 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846623a-3db4-4e22-994c-5f237944363f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.540128 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846623a-3db4-4e22-994c-5f237944363f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.540399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846623a-3db4-4e22-994c-5f237944363f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.541039 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846623a-3db4-4e22-994c-5f237944363f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.542640 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846623a-3db4-4e22-994c-5f237944363f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.552318 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1468/2923] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:46.556166 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.566424 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295054a-2f1f-4b52-a436-eafc196029fd admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.570931 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.571594 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] REQUEST_METHOD: `PUT` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.571869 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.572134 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] PATH_INFO: `/v3/projects/67ffae5d5f0743ab9720f1cfe3e6b19e/users/20efeab3f716454eaacf4ddca4272398/roles/87f1831257824560813f954348534540` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.586194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ffae5d5f0743ab9720f1cfe3e6b19e, user_id=20efeab3f716454eaacf4ddca4272398, role_id=87f1831257824560813f954348534540)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.589375 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.602796 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1456/2924] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:46 2022] PUT /identity/v3/projects/67ffae5d5f0743ab9720f1cfe3e6b19e/users/20efeab3f716454eaacf4ddca4272398/roles/87f1831257824560813f954348534540 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:46.614559 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.614781 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.615059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.642440 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] MFA Rules not processed for user `20efeab3f716454eaacf4ddca4272398`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:46.643697 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:46.708203 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1469/2925] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:46.712760 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.773354 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3759-7780-4760-b263-1ede82d9adef admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.778029 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.778616 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.778890 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.779176 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.779826 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.781687 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.808004 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1457/2926] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 01:01:46 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:46.812198 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.822922 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c621e5-6796-45ed-bc7a-32b79b6295e7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.878133 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.878665 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.878945 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.879217 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.879869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.881784 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.902584 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1470/2927] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 01:01:46 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 01:01:46.906646 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.917116 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60375f00-8b4d-4c79-ae37-949ecf0c1aa3 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.921759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.922339 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.922614 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.922881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.923547 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.925107 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.935843 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1458/2928] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:46.939813 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:46.949552 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e101dc-0ad1-48c7-98d6-f2f985c9973c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:46.954087 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:46.954751 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:46.955029 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:46.955298 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] PATH_INFO: `/v3/projects/ecbbd05f6e69445587e2e1ef467de1ce/users/9bfca145c05d49d2991b1cd0315c62fc/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:46.967204 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:46.970410 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd05f6e69445587e2e1ef467de1ce, user_id=9bfca145c05d49d2991b1cd0315c62fc, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:46.973541 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:46.986499 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1471/2929] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:46 2022] PUT /identity/v3/projects/ecbbd05f6e69445587e2e1ef467de1ce/users/9bfca145c05d49d2991b1cd0315c62fc/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:46.990336 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:47.000674 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4933-71f1-49a3-80b5-2a5cd4f14588 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:47.040277 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:47.056644 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817cbc2-bc6c-4bec-9240-61bac915b914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:47.058013 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817cbc2-bc6c-4bec-9240-61bac915b914 admin admin] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:47.058410 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817cbc2-bc6c-4bec-9240-61bac915b914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:47.058689 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817cbc2-bc6c-4bec-9240-61bac915b914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:47.059326 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817cbc2-bc6c-4bec-9240-61bac915b914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:47.060960 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817cbc2-bc6c-4bec-9240-61bac915b914 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:47.070574 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1459/2930] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 01:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 01:01:47.074660 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:47.084799 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094496f-79ec-4309-9ccc-a76cc3da6d21 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:47.089940 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:47.090565 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:47.090843 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:47.091112 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] PATH_INFO: `/v3/projects/ecbbd05f6e69445587e2e1ef467de1ce/users/9bfca145c05d49d2991b1cd0315c62fc/roles/971eb6603a5a4b8a80d419690dfbeb95` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:47.093059 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd05f6e69445587e2e1ef467de1ce, user_id=9bfca145c05d49d2991b1cd0315c62fc, role_id=971eb6603a5a4b8a80d419690dfbeb95)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:47.095690 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:47.106090 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1472/2931] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 01:01:47 2022] PUT /identity/v3/projects/ecbbd05f6e69445587e2e1ef467de1ce/users/9bfca145c05d49d2991b1cd0315c62fc/roles/971eb6603a5a4b8a80d419690dfbeb95 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:47.115840 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] REQUEST_METHOD: `POST` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:47.116124 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:47.116422 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:47.143772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.auth.core [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] MFA Rules not processed for user `9bfca145c05d49d2991b1cd0315c62fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81609) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:47.144896 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:47.210128 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1460/2932] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:47.221880 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:47.223591 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:47.280101 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e884e-0a6b-427d-8a62-a7f088f89d38 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:47.283715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:47.284310 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:47.284590 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:47.284861 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:47.341825 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:47.343255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:47.351935 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1473/2933] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:48.845138 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:48.846610 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:48.858274 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bfde6-8014-44d3-a58a-2460b618250c None None] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:48.861830 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:48.862419 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:48.862696 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:48.862966 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:48.918527 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:48.919910 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:48.928790 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1461/2934] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:49.477525 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:49.486577 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10678a4-0100-4807-96d0-69b693928a6e service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:49.490118 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff521b5-a218-4ee6-9568-f6c2900168bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7727b52cd17b41708b243dac7b4f2c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:49.490701 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff521b5-a218-4ee6-9568-f6c2900168bd service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:49.490974 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff521b5-a218-4ee6-9568-f6c2900168bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:49.491255 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff521b5-a218-4ee6-9568-f6c2900168bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:49.500455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff521b5-a218-4ee6-9568-f6c2900168bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:49.501860 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff521b5-a218-4ee6-9568-f6c2900168bd service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:49.511917 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1474/2935] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:53.360998 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:53.422050 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f2c40-3c9f-4f6e-bbde-7ea2970d4b04 service cinder] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:53.426573 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:53.427194 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:53.427472 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:53.427772 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] PATH_INFO: `/v3/users/86a4f6d6685b47fab6816b4fa5d4b519` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:53.428892 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a4f6d6685b47fab6816b4fa5d4b519)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:53.430815 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:53.435304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=100 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:53.479300 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a4f6d6685b47fab6816b4fa5d4b519'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:53.486752 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a4f6d6685b47fab6816b4fa5d4b519'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:53.487358 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a4f6d6685b47fab6816b4fa5d4b519'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:53.494079 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a4f6d6685b47fab6816b4fa5d4b519'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:53.494718 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a4f6d6685b47fab6816b4fa5d4b519'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:53.512012 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1462/2936] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:53 2022] DELETE /identity/v3/users/86a4f6d6685b47fab6816b4fa5d4b519 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:53.516549 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:53.517043 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:53.517157 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:53.536564 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] MFA Rules not processed for user `34b8b29876a54a4285fc37bd5edb4042`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 01:01:53.537817 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:53.595234 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1475/2937] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 01:01:53.604919 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:53.606319 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:53.661717 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76372d40-d2dd-4b25-bf9e-987a533b6355 admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:53.665332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24bf5028fe8d4a619897cf34ba3ff809', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:53.665923 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:53.666734 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:53.667005 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:53.676632 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:53.678047 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:53.688090 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1463/2938] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:01:53.946302 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:53.947794 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:53.960441 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454aa6-3b7e-4190-8ced-344357d4969e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:53.965778 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:53.966430 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:53.966704 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:53.966970 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] PATH_INFO: `/v3/projects/860fd9e3759345a18a6e1699bd9b4279` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:53.971504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] RBAC: Authorizing `identity:delete_project(project_id=860fd9e3759345a18a6e1699bd9b4279)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:53.971504 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:53.994024 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:54.028007 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860fd9e3759345a18a6e1699bd9b4279', 'request_id': 'req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.034153 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860fd9e3759345a18a6e1699bd9b4279', 'request_id': 'req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.043400 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860fd9e3759345a18a6e1699bd9b4279', 'request_id': 'req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.056115 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] The token cache is being invalidate because project 860fd9e3759345a18a6e1699bd9b4279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:54.056399 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.057061 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.058540 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1476/2939] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:53 2022] DELETE /identity/v3/projects/860fd9e3759345a18a6e1699bd9b4279 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:54.062679 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:54.072129 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adac81-bfe1-47f8-b8e6-555af70ec6ee service neutron] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:54.126431 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:54.126972 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:54.127246 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:54.127540 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] PATH_INFO: `/v3/users/20efeab3f716454eaacf4ddca4272398` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:54.128633 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] RBAC: Authorizing `identity:delete_user(user_id=20efeab3f716454eaacf4ddca4272398)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:54.130512 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:54.159759 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:54.178332 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20efeab3f716454eaacf4ddca4272398'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.187304 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20efeab3f716454eaacf4ddca4272398'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.187929 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20efeab3f716454eaacf4ddca4272398'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.200388 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20efeab3f716454eaacf4ddca4272398'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.201082 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20efeab3f716454eaacf4ddca4272398'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.219227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1464/2940] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:54 2022] DELETE /identity/v3/users/20efeab3f716454eaacf4ddca4272398 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:54.491063 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:54.492538 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:54.511291 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7493dd-c9cb-43ea-b3a5-3e1e8421643f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:54.571732 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:54.572489 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:54.572714 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:54.572987 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] PATH_INFO: `/v3/projects/67ffae5d5f0743ab9720f1cfe3e6b19e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:54.574117 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ffae5d5f0743ab9720f1cfe3e6b19e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:54.576078 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:54.621523 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ffae5d5f0743ab9720f1cfe3e6b19e', 'request_id': 'req-de6f21f1-fec9-4606-8070-6d4a08437417'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.627036 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ffae5d5f0743ab9720f1cfe3e6b19e', 'request_id': 'req-de6f21f1-fec9-4606-8070-6d4a08437417'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.634359 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ffae5d5f0743ab9720f1cfe3e6b19e', 'request_id': 'req-de6f21f1-fec9-4606-8070-6d4a08437417'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.645244 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] The token cache is being invalidate because project 67ffae5d5f0743ab9720f1cfe3e6b19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:54.645530 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.646156 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.647869 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1477/2941] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:54 2022] DELETE /identity/v3/projects/67ffae5d5f0743ab9720f1cfe3e6b19e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:54.652105 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:54.654044 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:54.672753 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae831-4795-408d-9b0b-7f603500dfff admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:54.744641 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:54.745262 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:54.745542 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:54.745822 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] PATH_INFO: `/v3/users/9bfca145c05d49d2991b1cd0315c62fc` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:54.746932 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfca145c05d49d2991b1cd0315c62fc)` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:54.748870 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:54.793793 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfca145c05d49d2991b1cd0315c62fc'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.801312 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfca145c05d49d2991b1cd0315c62fc'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.801962 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfca145c05d49d2991b1cd0315c62fc'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.808828 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfca145c05d49d2991b1cd0315c62fc'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.809459 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.notifications [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfca145c05d49d2991b1cd0315c62fc'} {{(pid=81609) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:54.826227 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1465/2942] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 01:01:54 2022] DELETE /identity/v3/users/9bfca145c05d49d2991b1cd0315c62fc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:01:55.095243 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:01:55.096738 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:01:55.110497 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f21f1-fec9-4606-8070-6d4a08437417 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:01:55.141229 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:01:55.169476 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34b8b29876a54a4285fc37bd5edb4042', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1704ba0a17f4363ab56f84c09fdea08', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:01:55.170102 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:01:55.170377 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:01:55.170719 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] PATH_INFO: `/v3/projects/ecbbd05f6e69445587e2e1ef467de1ce` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:01:55.171790 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbbd05f6e69445587e2e1ef467de1ce)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:01:55.173715 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:01:55.219535 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbbd05f6e69445587e2e1ef467de1ce', 'request_id': 'req-0989aeb5-ff6e-4565-ae9f-851c12ff022e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:55.224887 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbbd05f6e69445587e2e1ef467de1ce', 'request_id': 'req-0989aeb5-ff6e-4565-ae9f-851c12ff022e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:55.232717 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbbd05f6e69445587e2e1ef467de1ce', 'request_id': 'req-0989aeb5-ff6e-4565-ae9f-851c12ff022e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:55.243500 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] The token cache is being invalidate because project ecbbd05f6e69445587e2e1ef467de1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 01:01:55.243795 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:55.244992 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0989aeb5-ff6e-4565-ae9f-851c12ff022e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 01:01:55.246455 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1478/2943] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 01:01:55 2022] DELETE /identity/v3/projects/ecbbd05f6e69445587e2e1ef467de1ce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 01:02:04.822881 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:02:05.240597 dv-cinder-857528-2-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 01:02:56.289521 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Authenticating user token {{(pid=81609) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 01:02:56.290925 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:02:56.432524 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4784ff-8bf4-4fde-8d13-683245998e9a admin admin] Validating token access rules against request {{(pid=81609) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 01:02:56.489741 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce70c358cf0f46139f419cf894a97fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '217beeb34c094465ace3f6e1b3c979a4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81609) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 01:02:56.490265 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] REQUEST_METHOD: `GET` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 01:02:56.490539 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] SCRIPT_NAME: `/identity` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 01:02:56.490800 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81609) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 01:02:56.492526 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.fernet_utils [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81609) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 01:02:56.503905 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 01:02:56.505460 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187ac48-967f-4fe5-9b56-56133f5b4cc2 service placement] RBAC: Authorization granted {{(pid=81609) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 01:02:56.515891 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: [pid: 81609|app: 0|req: 1466/2944] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 01:02:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 01:03:06.511143 dv-cinder-857528-2-iscsi devstack@keystone.service[81609]: DEBUG dbcounter [-] [81609] Writing DB stats keystone:SELECT=89 {{(pid=81609) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}